2023-05-02 02:53:34,886 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure timeout: 13 mins 2023-05-02 02:53:35,148 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e48d2209-986f-dd5e-f2e4-019027d7bcac 2023-05-02 02:53:35,163 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=3, rsPorts=, rsClass=null, numDataNodes=3, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-05-02 02:53:35,163 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e48d2209-986f-dd5e-f2e4-019027d7bcac/cluster_9b21c39f-0840-34d4-ae73-45b092a6b580, deleteOnExit=true 2023-05-02 02:53:35,164 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-05-02 02:53:35,164 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e48d2209-986f-dd5e-f2e4-019027d7bcac/test.cache.data in system properties and HBase conf 2023-05-02 02:53:35,165 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e48d2209-986f-dd5e-f2e4-019027d7bcac/hadoop.tmp.dir in system properties and HBase conf 2023-05-02 02:53:35,165 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e48d2209-986f-dd5e-f2e4-019027d7bcac/hadoop.log.dir in system properties and HBase conf 2023-05-02 02:53:35,166 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e48d2209-986f-dd5e-f2e4-019027d7bcac/mapreduce.cluster.local.dir in system properties and HBase conf 2023-05-02 02:53:35,166 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e48d2209-986f-dd5e-f2e4-019027d7bcac/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-05-02 02:53:35,166 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-05-02 02:53:35,285 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-05-02 02:53:35,751 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-05-02 02:53:35,758 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e48d2209-986f-dd5e-f2e4-019027d7bcac/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-05-02 02:53:35,759 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e48d2209-986f-dd5e-f2e4-019027d7bcac/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-05-02 02:53:35,760 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e48d2209-986f-dd5e-f2e4-019027d7bcac/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-05-02 02:53:35,760 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e48d2209-986f-dd5e-f2e4-019027d7bcac/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-05-02 02:53:35,760 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e48d2209-986f-dd5e-f2e4-019027d7bcac/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-05-02 02:53:35,761 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e48d2209-986f-dd5e-f2e4-019027d7bcac/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-05-02 02:53:35,761 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e48d2209-986f-dd5e-f2e4-019027d7bcac/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-05-02 02:53:35,762 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e48d2209-986f-dd5e-f2e4-019027d7bcac/dfs.journalnode.edits.dir in system properties and HBase conf 2023-05-02 02:53:35,762 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e48d2209-986f-dd5e-f2e4-019027d7bcac/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-05-02 02:53:35,763 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e48d2209-986f-dd5e-f2e4-019027d7bcac/nfs.dump.dir in system properties and HBase conf 2023-05-02 02:53:35,763 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e48d2209-986f-dd5e-f2e4-019027d7bcac/java.io.tmpdir in system properties and HBase conf 2023-05-02 02:53:35,763 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e48d2209-986f-dd5e-f2e4-019027d7bcac/dfs.journalnode.edits.dir in system properties and HBase conf 2023-05-02 02:53:35,764 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e48d2209-986f-dd5e-f2e4-019027d7bcac/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-05-02 02:53:35,764 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e48d2209-986f-dd5e-f2e4-019027d7bcac/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-05-02 02:53:36,441 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-05-02 02:53:36,445 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-05-02 02:53:36,757 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-05-02 02:53:36,972 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-05-02 02:53:36,988 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-05-02 02:53:37,043 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-05-02 02:53:37,093 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e48d2209-986f-dd5e-f2e4-019027d7bcac/java.io.tmpdir/Jetty_localhost_43787_hdfs____xvemo1/webapp 2023-05-02 02:53:37,276 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:43787 2023-05-02 02:53:37,299 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-05-02 02:53:37,300 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-05-02 02:53:37,972 WARN [Listener at localhost/38649] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-05-02 02:53:38,059 WARN [Listener at localhost/38649] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-05-02 02:53:38,083 WARN [Listener at localhost/38649] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-05-02 02:53:38,091 INFO [Listener at localhost/38649] log.Slf4jLog(67): jetty-6.1.26 2023-05-02 02:53:38,098 INFO [Listener at localhost/38649] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e48d2209-986f-dd5e-f2e4-019027d7bcac/java.io.tmpdir/Jetty_localhost_35999_datanode____.h0pwde/webapp 2023-05-02 02:53:38,232 INFO [Listener at localhost/38649] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:35999 2023-05-02 02:53:38,662 WARN [Listener at localhost/36967] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-05-02 02:53:38,713 WARN [Listener at localhost/36967] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-05-02 02:53:38,720 WARN [Listener at localhost/36967] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-05-02 02:53:38,722 INFO [Listener at localhost/36967] log.Slf4jLog(67): jetty-6.1.26 2023-05-02 02:53:38,728 INFO [Listener at localhost/36967] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e48d2209-986f-dd5e-f2e4-019027d7bcac/java.io.tmpdir/Jetty_localhost_39931_datanode____.wu7m94/webapp 2023-05-02 02:53:38,831 INFO [Listener at localhost/36967] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:39931 2023-05-02 02:53:38,843 WARN [Listener at localhost/32987] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-05-02 02:53:38,874 WARN [Listener at localhost/32987] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-05-02 02:53:38,880 WARN [Listener at localhost/32987] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-05-02 02:53:38,882 INFO [Listener at localhost/32987] log.Slf4jLog(67): jetty-6.1.26 2023-05-02 02:53:38,888 INFO [Listener at localhost/32987] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e48d2209-986f-dd5e-f2e4-019027d7bcac/java.io.tmpdir/Jetty_localhost_43639_datanode____.2drom4/webapp 2023-05-02 02:53:39,013 INFO [Listener at localhost/32987] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:43639 2023-05-02 02:53:39,025 WARN [Listener at localhost/37119] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-05-02 02:53:39,264 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x92db44dbccb49ec6: Processing first storage report for DS-7474380a-b612-47cd-a5a8-fda8891d6af4 from datanode e5b41dfb-e591-48a4-b02d-848c5b0dba36 2023-05-02 02:53:39,266 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x92db44dbccb49ec6: from storage DS-7474380a-b612-47cd-a5a8-fda8891d6af4 node DatanodeRegistration(127.0.0.1:45341, datanodeUuid=e5b41dfb-e591-48a4-b02d-848c5b0dba36, infoPort=37147, infoSecurePort=0, ipcPort=37119, storageInfo=lv=-57;cid=testClusterID;nsid=934977279;c=1682996016523), blocks: 0, hasStaleStorage: true, processing time: 2 msecs, invalidatedBlocks: 0 2023-05-02 02:53:39,266 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x958ba14a9fdbb91b: Processing first storage report for DS-0296c9ee-7ff7-48ab-9156-52aba2f67e66 from datanode 12e74f69-84cc-44e8-ac7e-14ebfb93f6fc 2023-05-02 02:53:39,266 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x958ba14a9fdbb91b: from storage DS-0296c9ee-7ff7-48ab-9156-52aba2f67e66 node DatanodeRegistration(127.0.0.1:40593, datanodeUuid=12e74f69-84cc-44e8-ac7e-14ebfb93f6fc, infoPort=36065, infoSecurePort=0, ipcPort=32987, storageInfo=lv=-57;cid=testClusterID;nsid=934977279;c=1682996016523), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-05-02 02:53:39,266 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xf1912758b11ec6de: Processing first storage report for DS-6a3a9032-322c-41df-b7eb-ee8fbec214ae from datanode 2b091f00-29ea-476e-ba4f-8ff3dee54078 2023-05-02 02:53:39,267 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xf1912758b11ec6de: from storage DS-6a3a9032-322c-41df-b7eb-ee8fbec214ae node DatanodeRegistration(127.0.0.1:33641, datanodeUuid=2b091f00-29ea-476e-ba4f-8ff3dee54078, infoPort=33105, infoSecurePort=0, ipcPort=36967, storageInfo=lv=-57;cid=testClusterID;nsid=934977279;c=1682996016523), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-05-02 02:53:39,267 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x92db44dbccb49ec6: Processing first storage report for DS-d3798509-9bed-4db6-abb1-2eb58b053a18 from datanode e5b41dfb-e591-48a4-b02d-848c5b0dba36 2023-05-02 02:53:39,267 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x92db44dbccb49ec6: from storage DS-d3798509-9bed-4db6-abb1-2eb58b053a18 node DatanodeRegistration(127.0.0.1:45341, datanodeUuid=e5b41dfb-e591-48a4-b02d-848c5b0dba36, infoPort=37147, infoSecurePort=0, ipcPort=37119, storageInfo=lv=-57;cid=testClusterID;nsid=934977279;c=1682996016523), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-05-02 02:53:39,267 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x958ba14a9fdbb91b: Processing first storage report for DS-940e3d2a-5dc3-44b7-8f26-083dd9426a60 from datanode 12e74f69-84cc-44e8-ac7e-14ebfb93f6fc 2023-05-02 02:53:39,267 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x958ba14a9fdbb91b: from storage DS-940e3d2a-5dc3-44b7-8f26-083dd9426a60 node DatanodeRegistration(127.0.0.1:40593, datanodeUuid=12e74f69-84cc-44e8-ac7e-14ebfb93f6fc, infoPort=36065, infoSecurePort=0, ipcPort=32987, storageInfo=lv=-57;cid=testClusterID;nsid=934977279;c=1682996016523), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-05-02 02:53:39,267 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xf1912758b11ec6de: Processing first storage report for DS-c852ff67-1a96-4322-99c7-ffa76370ef7b from datanode 2b091f00-29ea-476e-ba4f-8ff3dee54078 2023-05-02 02:53:39,267 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xf1912758b11ec6de: from storage DS-c852ff67-1a96-4322-99c7-ffa76370ef7b node DatanodeRegistration(127.0.0.1:33641, datanodeUuid=2b091f00-29ea-476e-ba4f-8ff3dee54078, infoPort=33105, infoSecurePort=0, ipcPort=36967, storageInfo=lv=-57;cid=testClusterID;nsid=934977279;c=1682996016523), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-05-02 02:53:39,457 DEBUG [Listener at localhost/37119] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e48d2209-986f-dd5e-f2e4-019027d7bcac 2023-05-02 02:53:39,522 INFO [Listener at localhost/37119] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e48d2209-986f-dd5e-f2e4-019027d7bcac/cluster_9b21c39f-0840-34d4-ae73-45b092a6b580/zookeeper_0, clientPort=53331, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e48d2209-986f-dd5e-f2e4-019027d7bcac/cluster_9b21c39f-0840-34d4-ae73-45b092a6b580/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e48d2209-986f-dd5e-f2e4-019027d7bcac/cluster_9b21c39f-0840-34d4-ae73-45b092a6b580/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-05-02 02:53:39,536 INFO [Listener at localhost/37119] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=53331 2023-05-02 02:53:39,546 INFO [Listener at localhost/37119] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-05-02 02:53:39,549 INFO [Listener at localhost/37119] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-05-02 02:53:40,254 INFO [Listener at localhost/37119] util.FSUtils(471): Created version file at hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903 with version=8 2023-05-02 02:53:40,254 INFO [Listener at localhost/37119] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/hbase-staging 2023-05-02 02:53:40,590 INFO [Listener at localhost/37119] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-05-02 02:53:41,043 INFO [Listener at localhost/37119] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-05-02 02:53:41,075 INFO [Listener at localhost/37119] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-05-02 02:53:41,076 INFO [Listener at localhost/37119] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-05-02 02:53:41,077 INFO [Listener at localhost/37119] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-05-02 02:53:41,077 INFO [Listener at localhost/37119] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-05-02 02:53:41,077 INFO [Listener at localhost/37119] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-05-02 02:53:41,217 INFO [Listener at localhost/37119] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-05-02 02:53:41,293 DEBUG [Listener at localhost/37119] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-05-02 02:53:41,390 INFO [Listener at localhost/37119] ipc.NettyRpcServer(120): Bind to /172.31.14.131:41119 2023-05-02 02:53:41,400 INFO [Listener at localhost/37119] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-05-02 02:53:41,402 INFO [Listener at localhost/37119] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-05-02 02:53:41,423 INFO [Listener at localhost/37119] zookeeper.RecoverableZooKeeper(93): Process identifier=master:41119 connecting to ZooKeeper ensemble=127.0.0.1:53331 2023-05-02 02:53:41,462 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:411190x0, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-05-02 02:53:41,465 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:41119-0x101dd492af80000 connected 2023-05-02 02:53:41,492 DEBUG [Listener at localhost/37119] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-05-02 02:53:41,493 DEBUG [Listener at localhost/37119] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-05-02 02:53:41,497 DEBUG [Listener at localhost/37119] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-05-02 02:53:41,505 DEBUG [Listener at localhost/37119] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=41119 2023-05-02 02:53:41,505 DEBUG [Listener at localhost/37119] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=41119 2023-05-02 02:53:41,505 DEBUG [Listener at localhost/37119] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=41119 2023-05-02 02:53:41,506 DEBUG [Listener at localhost/37119] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=41119 2023-05-02 02:53:41,506 DEBUG [Listener at localhost/37119] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=41119 2023-05-02 02:53:41,512 INFO [Listener at localhost/37119] master.HMaster(444): hbase.rootdir=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903, hbase.cluster.distributed=false 2023-05-02 02:53:41,580 INFO [Listener at localhost/37119] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-05-02 02:53:41,580 INFO [Listener at localhost/37119] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-05-02 02:53:41,580 INFO [Listener at localhost/37119] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-05-02 02:53:41,580 INFO [Listener at localhost/37119] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-05-02 02:53:41,581 INFO [Listener at localhost/37119] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-05-02 02:53:41,581 INFO [Listener at localhost/37119] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-05-02 02:53:41,585 INFO [Listener at localhost/37119] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-05-02 02:53:41,588 INFO [Listener at localhost/37119] ipc.NettyRpcServer(120): Bind to /172.31.14.131:39081 2023-05-02 02:53:41,591 INFO [Listener at localhost/37119] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-05-02 02:53:41,597 DEBUG [Listener at localhost/37119] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-05-02 02:53:41,598 INFO [Listener at localhost/37119] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-05-02 02:53:41,600 INFO [Listener at localhost/37119] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-05-02 02:53:41,602 INFO [Listener at localhost/37119] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:39081 connecting to ZooKeeper ensemble=127.0.0.1:53331 2023-05-02 02:53:41,607 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:390810x0, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-05-02 02:53:41,608 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:39081-0x101dd492af80001 connected 2023-05-02 02:53:41,608 DEBUG [Listener at localhost/37119] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-05-02 02:53:41,609 DEBUG [Listener at localhost/37119] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-05-02 02:53:41,610 DEBUG [Listener at localhost/37119] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-05-02 02:53:41,611 DEBUG [Listener at localhost/37119] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=39081 2023-05-02 02:53:41,611 DEBUG [Listener at localhost/37119] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=39081 2023-05-02 02:53:41,612 DEBUG [Listener at localhost/37119] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=39081 2023-05-02 02:53:41,620 DEBUG [Listener at localhost/37119] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=39081 2023-05-02 02:53:41,620 DEBUG [Listener at localhost/37119] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=39081 2023-05-02 02:53:41,634 INFO [Listener at localhost/37119] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-05-02 02:53:41,634 INFO [Listener at localhost/37119] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-05-02 02:53:41,634 INFO [Listener at localhost/37119] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-05-02 02:53:41,635 INFO [Listener at localhost/37119] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-05-02 02:53:41,635 INFO [Listener at localhost/37119] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-05-02 02:53:41,635 INFO [Listener at localhost/37119] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-05-02 02:53:41,636 INFO [Listener at localhost/37119] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-05-02 02:53:41,637 INFO [Listener at localhost/37119] ipc.NettyRpcServer(120): Bind to /172.31.14.131:38925 2023-05-02 02:53:41,638 INFO [Listener at localhost/37119] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-05-02 02:53:41,639 DEBUG [Listener at localhost/37119] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-05-02 02:53:41,640 INFO [Listener at localhost/37119] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-05-02 02:53:41,642 INFO [Listener at localhost/37119] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-05-02 02:53:41,645 INFO [Listener at localhost/37119] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:38925 connecting to ZooKeeper ensemble=127.0.0.1:53331 2023-05-02 02:53:41,652 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:389250x0, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-05-02 02:53:41,653 DEBUG [Listener at localhost/37119] zookeeper.ZKUtil(164): regionserver:389250x0, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-05-02 02:53:41,653 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:38925-0x101dd492af80002 connected 2023-05-02 02:53:41,654 DEBUG [Listener at localhost/37119] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-05-02 02:53:41,655 DEBUG [Listener at localhost/37119] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-05-02 02:53:41,656 DEBUG [Listener at localhost/37119] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=38925 2023-05-02 02:53:41,656 DEBUG [Listener at localhost/37119] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=38925 2023-05-02 02:53:41,656 DEBUG [Listener at localhost/37119] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=38925 2023-05-02 02:53:41,664 DEBUG [Listener at localhost/37119] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=38925 2023-05-02 02:53:41,664 DEBUG [Listener at localhost/37119] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=38925 2023-05-02 02:53:41,677 INFO [Listener at localhost/37119] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-05-02 02:53:41,677 INFO [Listener at localhost/37119] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-05-02 02:53:41,678 INFO [Listener at localhost/37119] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-05-02 02:53:41,678 INFO [Listener at localhost/37119] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-05-02 02:53:41,678 INFO [Listener at localhost/37119] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-05-02 02:53:41,678 INFO [Listener at localhost/37119] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-05-02 02:53:41,678 INFO [Listener at localhost/37119] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-05-02 02:53:41,680 INFO [Listener at localhost/37119] ipc.NettyRpcServer(120): Bind to /172.31.14.131:42019 2023-05-02 02:53:41,680 INFO [Listener at localhost/37119] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-05-02 02:53:41,682 DEBUG [Listener at localhost/37119] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-05-02 02:53:41,683 INFO [Listener at localhost/37119] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-05-02 02:53:41,685 INFO [Listener at localhost/37119] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-05-02 02:53:41,686 INFO [Listener at localhost/37119] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:42019 connecting to ZooKeeper ensemble=127.0.0.1:53331 2023-05-02 02:53:41,691 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:420190x0, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-05-02 02:53:41,692 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:42019-0x101dd492af80003 connected 2023-05-02 02:53:41,692 DEBUG [Listener at localhost/37119] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-05-02 02:53:41,692 DEBUG [Listener at localhost/37119] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-05-02 02:53:41,693 DEBUG [Listener at localhost/37119] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-05-02 02:53:41,696 DEBUG [Listener at localhost/37119] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=42019 2023-05-02 02:53:41,696 DEBUG [Listener at localhost/37119] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=42019 2023-05-02 02:53:41,696 DEBUG [Listener at localhost/37119] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=42019 2023-05-02 02:53:41,697 DEBUG [Listener at localhost/37119] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=42019 2023-05-02 02:53:41,700 DEBUG [Listener at localhost/37119] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=42019 2023-05-02 02:53:41,704 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,41119,1682996020397 2023-05-02 02:53:41,716 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-05-02 02:53:41,718 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,41119,1682996020397 2023-05-02 02:53:41,736 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-05-02 02:53:41,736 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-05-02 02:53:41,736 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-05-02 02:53:41,736 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-05-02 02:53:41,736 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-05-02 02:53:41,738 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-05-02 02:53:41,740 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,41119,1682996020397 from backup master directory 2023-05-02 02:53:41,740 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-05-02 02:53:41,745 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,41119,1682996020397 2023-05-02 02:53:41,745 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-05-02 02:53:41,746 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-05-02 02:53:41,746 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,41119,1682996020397 2023-05-02 02:53:41,749 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-05-02 02:53:41,750 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-05-02 02:53:41,849 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/hbase.id with ID: 60b29913-ec00-4c90-b1b6-9a84da4b59de 2023-05-02 02:53:41,887 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-05-02 02:53:41,901 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-05-02 02:53:41,949 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x64a54fc5 to 127.0.0.1:53331 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-05-02 02:53:41,982 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3f367ab1, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-05-02 02:53:42,006 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-05-02 02:53:42,008 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-05-02 02:53:42,025 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-05-02 02:53:42,025 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-05-02 02:53:42,027 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-05-02 02:53:42,031 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-05-02 02:53:42,032 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-05-02 02:53:42,067 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/MasterData/data/master/store-tmp 2023-05-02 02:53:42,102 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-02 02:53:42,102 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-05-02 02:53:42,102 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-05-02 02:53:42,102 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-05-02 02:53:42,103 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 1 ms 2023-05-02 02:53:42,103 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-05-02 02:53:42,103 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-05-02 02:53:42,103 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-05-02 02:53:42,104 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/MasterData/WALs/jenkins-hbase4.apache.org,41119,1682996020397 2023-05-02 02:53:42,124 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C41119%2C1682996020397, suffix=, logDir=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/MasterData/WALs/jenkins-hbase4.apache.org,41119,1682996020397, archiveDir=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/MasterData/oldWALs, maxLogs=10 2023-05-02 02:53:42,179 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45341,DS-7474380a-b612-47cd-a5a8-fda8891d6af4,DISK] 2023-05-02 02:53:42,179 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40593,DS-0296c9ee-7ff7-48ab-9156-52aba2f67e66,DISK] 2023-05-02 02:53:42,179 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33641,DS-6a3a9032-322c-41df-b7eb-ee8fbec214ae,DISK] 2023-05-02 02:53:42,187 DEBUG [RS-EventLoopGroup-5-1] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-05-02 02:53:42,265 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/MasterData/WALs/jenkins-hbase4.apache.org,41119,1682996020397/jenkins-hbase4.apache.org%2C41119%2C1682996020397.1682996022134 2023-05-02 02:53:42,265 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40593,DS-0296c9ee-7ff7-48ab-9156-52aba2f67e66,DISK], DatanodeInfoWithStorage[127.0.0.1:45341,DS-7474380a-b612-47cd-a5a8-fda8891d6af4,DISK], DatanodeInfoWithStorage[127.0.0.1:33641,DS-6a3a9032-322c-41df-b7eb-ee8fbec214ae,DISK]] 2023-05-02 02:53:42,266 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-05-02 02:53:42,266 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-02 02:53:42,269 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-05-02 02:53:42,270 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-05-02 02:53:42,329 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-05-02 02:53:42,337 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-05-02 02:53:42,363 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-05-02 02:53:42,376 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-02 02:53:42,383 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-05-02 02:53:42,385 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-05-02 02:53:42,404 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-05-02 02:53:42,408 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-05-02 02:53:42,409 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68503275, jitterRate=0.020778343081474304}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-05-02 02:53:42,410 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-05-02 02:53:42,411 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-05-02 02:53:42,435 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-02 02:53:42,436 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-02 02:53:42,438 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-02 02:53:42,440 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-05-02 02:53:42,472 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 32 msec 2023-05-02 02:53:42,473 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(95): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-05-02 02:53:42,497 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-05-02 02:53:42,503 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-05-02 02:53:42,526 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-05-02 02:53:42,530 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-05-02 02:53:42,531 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-05-02 02:53:42,536 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-05-02 02:53:42,540 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-05-02 02:53:42,542 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-05-02 02:53:42,544 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-05-02 02:53:42,545 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-05-02 02:53:42,557 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-05-02 02:53:42,562 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-05-02 02:53:42,562 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-05-02 02:53:42,562 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-05-02 02:53:42,562 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-05-02 02:53:42,562 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-05-02 02:53:42,563 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,41119,1682996020397, sessionid=0x101dd492af80000, setting cluster-up flag (Was=false) 2023-05-02 02:53:42,582 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-05-02 02:53:42,587 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-05-02 02:53:42,589 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,41119,1682996020397 2023-05-02 02:53:42,594 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-05-02 02:53:42,599 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-05-02 02:53:42,605 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,41119,1682996020397 2023-05-02 02:53:42,608 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.hbase-snapshot/.tmp 2023-05-02 02:53:42,704 INFO [RS:1;jenkins-hbase4:38925] regionserver.HRegionServer(952): ClusterId : 60b29913-ec00-4c90-b1b6-9a84da4b59de 2023-05-02 02:53:42,705 INFO [RS:2;jenkins-hbase4:42019] regionserver.HRegionServer(952): ClusterId : 60b29913-ec00-4c90-b1b6-9a84da4b59de 2023-05-02 02:53:42,704 INFO [RS:0;jenkins-hbase4:39081] regionserver.HRegionServer(952): ClusterId : 60b29913-ec00-4c90-b1b6-9a84da4b59de 2023-05-02 02:53:42,711 DEBUG [RS:0;jenkins-hbase4:39081] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-05-02 02:53:42,711 DEBUG [RS:1;jenkins-hbase4:38925] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-05-02 02:53:42,711 DEBUG [RS:2;jenkins-hbase4:42019] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-05-02 02:53:42,718 DEBUG [RS:1;jenkins-hbase4:38925] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-05-02 02:53:42,718 DEBUG [RS:2;jenkins-hbase4:42019] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-05-02 02:53:42,718 DEBUG [RS:0;jenkins-hbase4:39081] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-05-02 02:53:42,718 DEBUG [RS:2;jenkins-hbase4:42019] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-05-02 02:53:42,718 DEBUG [RS:1;jenkins-hbase4:38925] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-05-02 02:53:42,718 DEBUG [RS:0;jenkins-hbase4:39081] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-05-02 02:53:42,722 DEBUG [RS:2;jenkins-hbase4:42019] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-05-02 02:53:42,722 DEBUG [RS:1;jenkins-hbase4:38925] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-05-02 02:53:42,722 DEBUG [RS:0;jenkins-hbase4:39081] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-05-02 02:53:42,724 DEBUG [RS:0;jenkins-hbase4:39081] zookeeper.ReadOnlyZKClient(139): Connect 0x1fc1cff8 to 127.0.0.1:53331 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-05-02 02:53:42,724 DEBUG [RS:1;jenkins-hbase4:38925] zookeeper.ReadOnlyZKClient(139): Connect 0x25d1d127 to 127.0.0.1:53331 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-05-02 02:53:42,724 DEBUG [RS:2;jenkins-hbase4:42019] zookeeper.ReadOnlyZKClient(139): Connect 0x6cbc9d46 to 127.0.0.1:53331 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-05-02 02:53:42,763 DEBUG [RS:2;jenkins-hbase4:42019] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@627b6ea1, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-05-02 02:53:42,763 DEBUG [RS:1;jenkins-hbase4:38925] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2b162bed, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-05-02 02:53:42,764 DEBUG [RS:2;jenkins-hbase4:42019] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6644f636, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-05-02 02:53:42,764 DEBUG [RS:1;jenkins-hbase4:38925] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2f29ba57, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-05-02 02:53:42,765 DEBUG [RS:0;jenkins-hbase4:39081] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@11063270, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-05-02 02:53:42,765 DEBUG [RS:0;jenkins-hbase4:39081] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1539a488, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-05-02 02:53:42,766 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-05-02 02:53:42,776 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-05-02 02:53:42,776 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-05-02 02:53:42,777 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-05-02 02:53:42,777 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-05-02 02:53:42,777 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-05-02 02:53:42,777 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-02 02:53:42,777 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-05-02 02:53:42,777 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-02 02:53:42,778 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682996052778 2023-05-02 02:53:42,780 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-05-02 02:53:42,787 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-05-02 02:53:42,788 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-05-02 02:53:42,791 DEBUG [RS:2;jenkins-hbase4:42019] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:2;jenkins-hbase4:42019 2023-05-02 02:53:42,791 DEBUG [RS:1;jenkins-hbase4:38925] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase4:38925 2023-05-02 02:53:42,793 DEBUG [RS:0;jenkins-hbase4:39081] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:39081 2023-05-02 02:53:42,793 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-05-02 02:53:42,794 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-05-02 02:53:42,796 INFO [RS:2;jenkins-hbase4:42019] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-05-02 02:53:42,796 INFO [RS:0;jenkins-hbase4:39081] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-05-02 02:53:42,797 INFO [RS:2;jenkins-hbase4:42019] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-05-02 02:53:42,796 INFO [RS:1;jenkins-hbase4:38925] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-05-02 02:53:42,797 DEBUG [RS:2;jenkins-hbase4:42019] regionserver.HRegionServer(1023): About to register with Master. 2023-05-02 02:53:42,797 INFO [RS:0;jenkins-hbase4:39081] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-05-02 02:53:42,797 INFO [RS:1;jenkins-hbase4:38925] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-05-02 02:53:42,798 DEBUG [RS:0;jenkins-hbase4:39081] regionserver.HRegionServer(1023): About to register with Master. 2023-05-02 02:53:42,798 DEBUG [RS:1;jenkins-hbase4:38925] regionserver.HRegionServer(1023): About to register with Master. 2023-05-02 02:53:42,800 INFO [RS:1;jenkins-hbase4:38925] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,41119,1682996020397 with isa=jenkins-hbase4.apache.org/172.31.14.131:38925, startcode=1682996021633 2023-05-02 02:53:42,800 INFO [RS:2;jenkins-hbase4:42019] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,41119,1682996020397 with isa=jenkins-hbase4.apache.org/172.31.14.131:42019, startcode=1682996021676 2023-05-02 02:53:42,800 INFO [RS:0;jenkins-hbase4:39081] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,41119,1682996020397 with isa=jenkins-hbase4.apache.org/172.31.14.131:39081, startcode=1682996021579 2023-05-02 02:53:42,801 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-05-02 02:53:42,801 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-05-02 02:53:42,802 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-05-02 02:53:42,802 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-05-02 02:53:42,802 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-05-02 02:53:42,804 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-05-02 02:53:42,805 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-05-02 02:53:42,806 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-05-02 02:53:42,811 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-05-02 02:53:42,811 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-05-02 02:53:42,814 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1682996022813,5,FailOnTimeoutGroup] 2023-05-02 02:53:42,815 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1682996022814,5,FailOnTimeoutGroup] 2023-05-02 02:53:42,815 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-05-02 02:53:42,815 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-05-02 02:53:42,817 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-05-02 02:53:42,817 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-05-02 02:53:42,820 DEBUG [RS:0;jenkins-hbase4:39081] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-05-02 02:53:42,820 DEBUG [RS:1;jenkins-hbase4:38925] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-05-02 02:53:42,820 DEBUG [RS:2;jenkins-hbase4:42019] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-05-02 02:53:42,859 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-05-02 02:53:42,860 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-05-02 02:53:42,861 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903 2023-05-02 02:53:42,905 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50585, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=RegionServerStatusService 2023-05-02 02:53:42,905 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42147, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=RegionServerStatusService 2023-05-02 02:53:42,906 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-02 02:53:42,905 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:46919, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-05-02 02:53:42,909 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-05-02 02:53:42,912 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/hbase/meta/1588230740/info 2023-05-02 02:53:42,912 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-05-02 02:53:42,913 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-02 02:53:42,914 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-05-02 02:53:42,916 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/hbase/meta/1588230740/rep_barrier 2023-05-02 02:53:42,917 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-05-02 02:53:42,918 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-02 02:53:42,918 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-05-02 02:53:42,920 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:42,920 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41119] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:42,921 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/hbase/meta/1588230740/table 2023-05-02 02:53:42,921 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=41119] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:42,922 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-05-02 02:53:42,923 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-02 02:53:42,925 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/hbase/meta/1588230740 2023-05-02 02:53:42,926 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/hbase/meta/1588230740 2023-05-02 02:53:42,930 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-05-02 02:53:42,933 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-05-02 02:53:42,937 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-05-02 02:53:42,939 DEBUG [RS:1;jenkins-hbase4:38925] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903 2023-05-02 02:53:42,939 DEBUG [RS:2;jenkins-hbase4:42019] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903 2023-05-02 02:53:42,939 DEBUG [RS:0;jenkins-hbase4:39081] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903 2023-05-02 02:53:42,939 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72735338, jitterRate=0.0838409960269928}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-05-02 02:53:42,939 DEBUG [RS:0;jenkins-hbase4:39081] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:38649 2023-05-02 02:53:42,939 DEBUG [RS:2;jenkins-hbase4:42019] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:38649 2023-05-02 02:53:42,939 DEBUG [RS:1;jenkins-hbase4:38925] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:38649 2023-05-02 02:53:42,940 DEBUG [RS:2;jenkins-hbase4:42019] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-05-02 02:53:42,939 DEBUG [RS:0;jenkins-hbase4:39081] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-05-02 02:53:42,939 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-05-02 02:53:42,940 DEBUG [RS:1;jenkins-hbase4:38925] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-05-02 02:53:42,940 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-05-02 02:53:42,940 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-05-02 02:53:42,940 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-05-02 02:53:42,940 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-05-02 02:53:42,940 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-05-02 02:53:42,941 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-05-02 02:53:42,942 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-05-02 02:53:42,947 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-05-02 02:53:42,947 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-05-02 02:53:42,947 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-05-02 02:53:42,951 DEBUG [RS:1;jenkins-hbase4:38925] zookeeper.ZKUtil(162): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:42,951 WARN [RS:1;jenkins-hbase4:38925] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-05-02 02:53:42,951 INFO [RS:1;jenkins-hbase4:38925] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-05-02 02:53:42,951 DEBUG [RS:0;jenkins-hbase4:39081] zookeeper.ZKUtil(162): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:42,952 DEBUG [RS:1;jenkins-hbase4:38925] regionserver.HRegionServer(1947): logDir=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/WALs/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:42,952 WARN [RS:0;jenkins-hbase4:39081] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-05-02 02:53:42,951 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,39081,1682996021579] 2023-05-02 02:53:42,952 INFO [RS:0;jenkins-hbase4:39081] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-05-02 02:53:42,953 DEBUG [RS:0;jenkins-hbase4:39081] regionserver.HRegionServer(1947): logDir=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/WALs/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:42,953 DEBUG [RS:2;jenkins-hbase4:42019] zookeeper.ZKUtil(162): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:42,953 WARN [RS:2;jenkins-hbase4:42019] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-05-02 02:53:42,953 INFO [RS:2;jenkins-hbase4:42019] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-05-02 02:53:42,954 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,38925,1682996021633] 2023-05-02 02:53:42,954 DEBUG [RS:2;jenkins-hbase4:42019] regionserver.HRegionServer(1947): logDir=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/WALs/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:42,955 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,42019,1682996021676] 2023-05-02 02:53:42,958 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-05-02 02:53:42,970 DEBUG [RS:0;jenkins-hbase4:39081] zookeeper.ZKUtil(162): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:42,970 DEBUG [RS:1;jenkins-hbase4:38925] zookeeper.ZKUtil(162): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:42,971 DEBUG [RS:2;jenkins-hbase4:42019] zookeeper.ZKUtil(162): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:42,971 DEBUG [RS:1;jenkins-hbase4:38925] zookeeper.ZKUtil(162): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:42,971 DEBUG [RS:0;jenkins-hbase4:39081] zookeeper.ZKUtil(162): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:42,971 DEBUG [RS:2;jenkins-hbase4:42019] zookeeper.ZKUtil(162): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:42,972 DEBUG [RS:1;jenkins-hbase4:38925] zookeeper.ZKUtil(162): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:42,972 DEBUG [RS:0;jenkins-hbase4:39081] zookeeper.ZKUtil(162): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:42,972 DEBUG [RS:2;jenkins-hbase4:42019] zookeeper.ZKUtil(162): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:42,980 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-05-02 02:53:42,982 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-05-02 02:53:42,984 DEBUG [RS:0;jenkins-hbase4:39081] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-05-02 02:53:42,985 DEBUG [RS:1;jenkins-hbase4:38925] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-05-02 02:53:42,984 DEBUG [RS:2;jenkins-hbase4:42019] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-05-02 02:53:42,994 INFO [RS:2;jenkins-hbase4:42019] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-05-02 02:53:42,994 INFO [RS:1;jenkins-hbase4:38925] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-05-02 02:53:42,994 INFO [RS:0;jenkins-hbase4:39081] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-05-02 02:53:43,016 INFO [RS:2;jenkins-hbase4:42019] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-05-02 02:53:43,016 INFO [RS:1;jenkins-hbase4:38925] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-05-02 02:53:43,017 INFO [RS:0;jenkins-hbase4:39081] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-05-02 02:53:43,021 INFO [RS:2;jenkins-hbase4:42019] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-05-02 02:53:43,021 INFO [RS:1;jenkins-hbase4:38925] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-05-02 02:53:43,021 INFO [RS:0;jenkins-hbase4:39081] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-05-02 02:53:43,021 INFO [RS:1;jenkins-hbase4:38925] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-05-02 02:53:43,021 INFO [RS:2;jenkins-hbase4:42019] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-05-02 02:53:43,021 INFO [RS:0;jenkins-hbase4:39081] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-05-02 02:53:43,023 INFO [RS:1;jenkins-hbase4:38925] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-05-02 02:53:43,023 INFO [RS:0;jenkins-hbase4:39081] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-05-02 02:53:43,023 INFO [RS:2;jenkins-hbase4:42019] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-05-02 02:53:43,030 INFO [RS:2;jenkins-hbase4:42019] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-05-02 02:53:43,030 INFO [RS:0;jenkins-hbase4:39081] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-05-02 02:53:43,030 INFO [RS:1;jenkins-hbase4:38925] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-05-02 02:53:43,031 DEBUG [RS:2;jenkins-hbase4:42019] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-02 02:53:43,031 DEBUG [RS:0;jenkins-hbase4:39081] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-02 02:53:43,031 DEBUG [RS:2;jenkins-hbase4:42019] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-02 02:53:43,031 DEBUG [RS:0;jenkins-hbase4:39081] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-02 02:53:43,031 DEBUG [RS:2;jenkins-hbase4:42019] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-02 02:53:43,031 DEBUG [RS:0;jenkins-hbase4:39081] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-02 02:53:43,031 DEBUG [RS:2;jenkins-hbase4:42019] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-02 02:53:43,031 DEBUG [RS:0;jenkins-hbase4:39081] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-02 02:53:43,032 DEBUG [RS:2;jenkins-hbase4:42019] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-02 02:53:43,032 DEBUG [RS:0;jenkins-hbase4:39081] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-02 02:53:43,032 DEBUG [RS:2;jenkins-hbase4:42019] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-05-02 02:53:43,032 DEBUG [RS:0;jenkins-hbase4:39081] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-05-02 02:53:43,032 DEBUG [RS:2;jenkins-hbase4:42019] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-02 02:53:43,032 DEBUG [RS:0;jenkins-hbase4:39081] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-02 02:53:43,031 DEBUG [RS:1;jenkins-hbase4:38925] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-02 02:53:43,032 DEBUG [RS:0;jenkins-hbase4:39081] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-02 02:53:43,032 DEBUG [RS:2;jenkins-hbase4:42019] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-02 02:53:43,032 DEBUG [RS:0;jenkins-hbase4:39081] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-02 02:53:43,032 DEBUG [RS:1;jenkins-hbase4:38925] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-02 02:53:43,032 DEBUG [RS:0;jenkins-hbase4:39081] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-02 02:53:43,032 DEBUG [RS:2;jenkins-hbase4:42019] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-02 02:53:43,032 DEBUG [RS:1;jenkins-hbase4:38925] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-02 02:53:43,032 DEBUG [RS:2;jenkins-hbase4:42019] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-02 02:53:43,033 DEBUG [RS:1;jenkins-hbase4:38925] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-02 02:53:43,033 DEBUG [RS:1;jenkins-hbase4:38925] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-02 02:53:43,033 DEBUG [RS:1;jenkins-hbase4:38925] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-05-02 02:53:43,033 DEBUG [RS:1;jenkins-hbase4:38925] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-02 02:53:43,033 DEBUG [RS:1;jenkins-hbase4:38925] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-02 02:53:43,034 DEBUG [RS:1;jenkins-hbase4:38925] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-02 02:53:43,034 DEBUG [RS:1;jenkins-hbase4:38925] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-02 02:53:43,035 INFO [RS:0;jenkins-hbase4:39081] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-05-02 02:53:43,036 INFO [RS:2;jenkins-hbase4:42019] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-05-02 02:53:43,036 INFO [RS:0;jenkins-hbase4:39081] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-05-02 02:53:43,036 INFO [RS:2;jenkins-hbase4:42019] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-05-02 02:53:43,036 INFO [RS:0;jenkins-hbase4:39081] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-05-02 02:53:43,036 INFO [RS:2;jenkins-hbase4:42019] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-05-02 02:53:43,040 INFO [RS:1;jenkins-hbase4:38925] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-05-02 02:53:43,040 INFO [RS:1;jenkins-hbase4:38925] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-05-02 02:53:43,040 INFO [RS:1;jenkins-hbase4:38925] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-05-02 02:53:43,055 INFO [RS:2;jenkins-hbase4:42019] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-05-02 02:53:43,057 INFO [RS:0;jenkins-hbase4:39081] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-05-02 02:53:43,057 INFO [RS:1;jenkins-hbase4:38925] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-05-02 02:53:43,058 INFO [RS:0;jenkins-hbase4:39081] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39081,1682996021579-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-05-02 02:53:43,058 INFO [RS:2;jenkins-hbase4:42019] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,42019,1682996021676-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-05-02 02:53:43,058 INFO [RS:1;jenkins-hbase4:38925] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38925,1682996021633-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-05-02 02:53:43,077 INFO [RS:0;jenkins-hbase4:39081] regionserver.Replication(203): jenkins-hbase4.apache.org,39081,1682996021579 started 2023-05-02 02:53:43,077 INFO [RS:0;jenkins-hbase4:39081] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,39081,1682996021579, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:39081, sessionid=0x101dd492af80001 2023-05-02 02:53:43,077 DEBUG [RS:0;jenkins-hbase4:39081] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-05-02 02:53:43,077 DEBUG [RS:0;jenkins-hbase4:39081] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:43,077 DEBUG [RS:0;jenkins-hbase4:39081] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,39081,1682996021579' 2023-05-02 02:53:43,078 DEBUG [RS:0;jenkins-hbase4:39081] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:43,078 DEBUG [RS:0;jenkins-hbase4:39081] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:43,079 DEBUG [RS:0;jenkins-hbase4:39081] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-05-02 02:53:43,079 DEBUG [RS:0;jenkins-hbase4:39081] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-05-02 02:53:43,079 DEBUG [RS:0;jenkins-hbase4:39081] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:43,079 DEBUG [RS:0;jenkins-hbase4:39081] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,39081,1682996021579' 2023-05-02 02:53:43,079 DEBUG [RS:0;jenkins-hbase4:39081] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-05-02 02:53:43,080 DEBUG [RS:0;jenkins-hbase4:39081] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-05-02 02:53:43,080 DEBUG [RS:0;jenkins-hbase4:39081] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-05-02 02:53:43,080 INFO [RS:0;jenkins-hbase4:39081] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-05-02 02:53:43,080 INFO [RS:0;jenkins-hbase4:39081] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-05-02 02:53:43,080 INFO [RS:2;jenkins-hbase4:42019] regionserver.Replication(203): jenkins-hbase4.apache.org,42019,1682996021676 started 2023-05-02 02:53:43,081 INFO [RS:2;jenkins-hbase4:42019] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,42019,1682996021676, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:42019, sessionid=0x101dd492af80003 2023-05-02 02:53:43,084 INFO [RS:1;jenkins-hbase4:38925] regionserver.Replication(203): jenkins-hbase4.apache.org,38925,1682996021633 started 2023-05-02 02:53:43,084 DEBUG [RS:2;jenkins-hbase4:42019] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-05-02 02:53:43,084 INFO [RS:1;jenkins-hbase4:38925] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,38925,1682996021633, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:38925, sessionid=0x101dd492af80002 2023-05-02 02:53:43,084 DEBUG [RS:2;jenkins-hbase4:42019] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:43,084 DEBUG [RS:1;jenkins-hbase4:38925] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-05-02 02:53:43,085 DEBUG [RS:1;jenkins-hbase4:38925] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:43,085 DEBUG [RS:1;jenkins-hbase4:38925] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,38925,1682996021633' 2023-05-02 02:53:43,085 DEBUG [RS:1;jenkins-hbase4:38925] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:43,084 DEBUG [RS:2;jenkins-hbase4:42019] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,42019,1682996021676' 2023-05-02 02:53:43,085 DEBUG [RS:2;jenkins-hbase4:42019] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:43,085 DEBUG [RS:1;jenkins-hbase4:38925] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:43,085 DEBUG [RS:2;jenkins-hbase4:42019] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:43,086 DEBUG [RS:1;jenkins-hbase4:38925] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-05-02 02:53:43,086 DEBUG [RS:2;jenkins-hbase4:42019] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-05-02 02:53:43,086 DEBUG [RS:1;jenkins-hbase4:38925] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-05-02 02:53:43,086 DEBUG [RS:2;jenkins-hbase4:42019] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-05-02 02:53:43,086 DEBUG [RS:1;jenkins-hbase4:38925] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:43,086 DEBUG [RS:1;jenkins-hbase4:38925] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,38925,1682996021633' 2023-05-02 02:53:43,086 DEBUG [RS:1;jenkins-hbase4:38925] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-05-02 02:53:43,086 DEBUG [RS:2;jenkins-hbase4:42019] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:43,087 DEBUG [RS:2;jenkins-hbase4:42019] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,42019,1682996021676' 2023-05-02 02:53:43,087 DEBUG [RS:2;jenkins-hbase4:42019] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-05-02 02:53:43,087 DEBUG [RS:1;jenkins-hbase4:38925] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-05-02 02:53:43,087 DEBUG [RS:2;jenkins-hbase4:42019] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-05-02 02:53:43,087 DEBUG [RS:1;jenkins-hbase4:38925] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-05-02 02:53:43,088 INFO [RS:1;jenkins-hbase4:38925] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-05-02 02:53:43,088 INFO [RS:1;jenkins-hbase4:38925] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-05-02 02:53:43,088 DEBUG [RS:2;jenkins-hbase4:42019] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-05-02 02:53:43,088 INFO [RS:2;jenkins-hbase4:42019] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-05-02 02:53:43,089 INFO [RS:2;jenkins-hbase4:42019] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-05-02 02:53:43,135 DEBUG [jenkins-hbase4:41119] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=3, allServersCount=3 2023-05-02 02:53:43,138 DEBUG [jenkins-hbase4:41119] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-05-02 02:53:43,145 DEBUG [jenkins-hbase4:41119] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-05-02 02:53:43,145 DEBUG [jenkins-hbase4:41119] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-05-02 02:53:43,145 DEBUG [jenkins-hbase4:41119] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-05-02 02:53:43,145 DEBUG [jenkins-hbase4:41119] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-05-02 02:53:43,148 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,38925,1682996021633, state=OPENING 2023-05-02 02:53:43,156 DEBUG [PEWorker-1] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-05-02 02:53:43,157 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-05-02 02:53:43,157 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-05-02 02:53:43,162 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,38925,1682996021633}] 2023-05-02 02:53:43,195 INFO [RS:2;jenkins-hbase4:42019] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C42019%2C1682996021676, suffix=, logDir=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/WALs/jenkins-hbase4.apache.org,42019,1682996021676, archiveDir=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/oldWALs, maxLogs=32 2023-05-02 02:53:43,195 INFO [RS:0;jenkins-hbase4:39081] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C39081%2C1682996021579, suffix=, logDir=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/WALs/jenkins-hbase4.apache.org,39081,1682996021579, archiveDir=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/oldWALs, maxLogs=32 2023-05-02 02:53:43,195 INFO [RS:1;jenkins-hbase4:38925] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C38925%2C1682996021633, suffix=, logDir=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/WALs/jenkins-hbase4.apache.org,38925,1682996021633, archiveDir=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/oldWALs, maxLogs=32 2023-05-02 02:53:43,232 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33641,DS-6a3a9032-322c-41df-b7eb-ee8fbec214ae,DISK] 2023-05-02 02:53:43,233 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33641,DS-6a3a9032-322c-41df-b7eb-ee8fbec214ae,DISK] 2023-05-02 02:53:43,235 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40593,DS-0296c9ee-7ff7-48ab-9156-52aba2f67e66,DISK] 2023-05-02 02:53:43,235 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40593,DS-0296c9ee-7ff7-48ab-9156-52aba2f67e66,DISK] 2023-05-02 02:53:43,236 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45341,DS-7474380a-b612-47cd-a5a8-fda8891d6af4,DISK] 2023-05-02 02:53:43,236 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45341,DS-7474380a-b612-47cd-a5a8-fda8891d6af4,DISK] 2023-05-02 02:53:43,236 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33641,DS-6a3a9032-322c-41df-b7eb-ee8fbec214ae,DISK] 2023-05-02 02:53:43,236 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45341,DS-7474380a-b612-47cd-a5a8-fda8891d6af4,DISK] 2023-05-02 02:53:43,237 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40593,DS-0296c9ee-7ff7-48ab-9156-52aba2f67e66,DISK] 2023-05-02 02:53:43,250 INFO [RS:0;jenkins-hbase4:39081] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/WALs/jenkins-hbase4.apache.org,39081,1682996021579/jenkins-hbase4.apache.org%2C39081%2C1682996021579.1682996023201 2023-05-02 02:53:43,251 INFO [RS:1;jenkins-hbase4:38925] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/WALs/jenkins-hbase4.apache.org,38925,1682996021633/jenkins-hbase4.apache.org%2C38925%2C1682996021633.1682996023201 2023-05-02 02:53:43,251 INFO [RS:2;jenkins-hbase4:42019] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/WALs/jenkins-hbase4.apache.org,42019,1682996021676/jenkins-hbase4.apache.org%2C42019%2C1682996021676.1682996023201 2023-05-02 02:53:43,251 DEBUG [RS:0;jenkins-hbase4:39081] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40593,DS-0296c9ee-7ff7-48ab-9156-52aba2f67e66,DISK], DatanodeInfoWithStorage[127.0.0.1:45341,DS-7474380a-b612-47cd-a5a8-fda8891d6af4,DISK], DatanodeInfoWithStorage[127.0.0.1:33641,DS-6a3a9032-322c-41df-b7eb-ee8fbec214ae,DISK]] 2023-05-02 02:53:43,251 DEBUG [RS:2;jenkins-hbase4:42019] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40593,DS-0296c9ee-7ff7-48ab-9156-52aba2f67e66,DISK], DatanodeInfoWithStorage[127.0.0.1:45341,DS-7474380a-b612-47cd-a5a8-fda8891d6af4,DISK], DatanodeInfoWithStorage[127.0.0.1:33641,DS-6a3a9032-322c-41df-b7eb-ee8fbec214ae,DISK]] 2023-05-02 02:53:43,255 DEBUG [RS:1;jenkins-hbase4:38925] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:33641,DS-6a3a9032-322c-41df-b7eb-ee8fbec214ae,DISK], DatanodeInfoWithStorage[127.0.0.1:40593,DS-0296c9ee-7ff7-48ab-9156-52aba2f67e66,DISK], DatanodeInfoWithStorage[127.0.0.1:45341,DS-7474380a-b612-47cd-a5a8-fda8891d6af4,DISK]] 2023-05-02 02:53:43,350 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:43,353 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-05-02 02:53:43,357 INFO [RS-EventLoopGroup-4-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:35096, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-05-02 02:53:43,370 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-05-02 02:53:43,370 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-05-02 02:53:43,374 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C38925%2C1682996021633.meta, suffix=.meta, logDir=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/WALs/jenkins-hbase4.apache.org,38925,1682996021633, archiveDir=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/oldWALs, maxLogs=32 2023-05-02 02:53:43,397 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33641,DS-6a3a9032-322c-41df-b7eb-ee8fbec214ae,DISK] 2023-05-02 02:53:43,397 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45341,DS-7474380a-b612-47cd-a5a8-fda8891d6af4,DISK] 2023-05-02 02:53:43,397 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40593,DS-0296c9ee-7ff7-48ab-9156-52aba2f67e66,DISK] 2023-05-02 02:53:43,402 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/WALs/jenkins-hbase4.apache.org,38925,1682996021633/jenkins-hbase4.apache.org%2C38925%2C1682996021633.meta.1682996023375.meta 2023-05-02 02:53:43,402 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40593,DS-0296c9ee-7ff7-48ab-9156-52aba2f67e66,DISK], DatanodeInfoWithStorage[127.0.0.1:45341,DS-7474380a-b612-47cd-a5a8-fda8891d6af4,DISK], DatanodeInfoWithStorage[127.0.0.1:33641,DS-6a3a9032-322c-41df-b7eb-ee8fbec214ae,DISK]] 2023-05-02 02:53:43,403 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-05-02 02:53:43,407 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-02 02:53:43,408 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-05-02 02:53:43,424 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-05-02 02:53:43,427 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-05-02 02:53:43,431 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-05-02 02:53:43,432 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-02 02:53:43,432 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-05-02 02:53:43,432 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-05-02 02:53:43,435 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-05-02 02:53:43,438 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/hbase/meta/1588230740/info 2023-05-02 02:53:43,438 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/hbase/meta/1588230740/info 2023-05-02 02:53:43,439 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-05-02 02:53:43,440 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-02 02:53:43,440 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-05-02 02:53:43,442 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/hbase/meta/1588230740/rep_barrier 2023-05-02 02:53:43,442 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/hbase/meta/1588230740/rep_barrier 2023-05-02 02:53:43,442 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-05-02 02:53:43,443 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-02 02:53:43,443 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-05-02 02:53:43,445 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/hbase/meta/1588230740/table 2023-05-02 02:53:43,445 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/hbase/meta/1588230740/table 2023-05-02 02:53:43,446 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-05-02 02:53:43,446 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-02 02:53:43,449 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/hbase/meta/1588230740 2023-05-02 02:53:43,452 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/hbase/meta/1588230740 2023-05-02 02:53:43,456 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-05-02 02:53:43,458 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-05-02 02:53:43,459 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59947795, jitterRate=-0.10670824348926544}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-05-02 02:53:43,459 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-05-02 02:53:43,472 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1682996023342 2023-05-02 02:53:43,492 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:meta,,1.1588230740 2023-05-02 02:53:43,493 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-05-02 02:53:43,494 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,38925,1682996021633, state=OPEN 2023-05-02 02:53:43,496 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-05-02 02:53:43,496 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-05-02 02:53:43,501 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-05-02 02:53:43,501 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,38925,1682996021633 in 334 msec 2023-05-02 02:53:43,506 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-05-02 02:53:43,506 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 544 msec 2023-05-02 02:53:43,512 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 849 msec 2023-05-02 02:53:43,512 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1682996023512, completionTime=-1 2023-05-02 02:53:43,512 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=3; waited=0ms, expected min=3 server(s), max=3 server(s), master is running 2023-05-02 02:53:43,512 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-05-02 02:53:43,574 DEBUG [hconnection-0x1f586e56-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-05-02 02:53:43,577 INFO [RS-EventLoopGroup-4-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:57930, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-05-02 02:53:43,596 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-02 02:53:43,596 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682996083596 2023-05-02 02:53:43,597 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682996143597 2023-05-02 02:53:43,597 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 84 msec 2023-05-02 02:53:43,623 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,41119,1682996020397-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-05-02 02:53:43,624 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,41119,1682996020397-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-05-02 02:53:43,624 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,41119,1682996020397-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-05-02 02:53:43,625 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:41119, period=300000, unit=MILLISECONDS is enabled. 2023-05-02 02:53:43,626 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-05-02 02:53:43,635 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-05-02 02:53:43,643 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-05-02 02:53:43,644 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-05-02 02:53:43,653 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-05-02 02:53:43,656 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-05-02 02:53:43,659 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-05-02 02:53:43,682 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/hbase/namespace/5c823d6f15ad7b8fe82f9ae4beee6513 2023-05-02 02:53:43,684 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/hbase/namespace/5c823d6f15ad7b8fe82f9ae4beee6513 empty. 2023-05-02 02:53:43,686 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/hbase/namespace/5c823d6f15ad7b8fe82f9ae4beee6513 2023-05-02 02:53:43,686 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-05-02 02:53:43,726 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-05-02 02:53:43,728 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => 5c823d6f15ad7b8fe82f9ae4beee6513, NAME => 'hbase:namespace,,1682996023643.5c823d6f15ad7b8fe82f9ae4beee6513.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp 2023-05-02 02:53:43,745 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1682996023643.5c823d6f15ad7b8fe82f9ae4beee6513.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-02 02:53:43,745 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing 5c823d6f15ad7b8fe82f9ae4beee6513, disabling compactions & flushes 2023-05-02 02:53:43,745 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1682996023643.5c823d6f15ad7b8fe82f9ae4beee6513. 2023-05-02 02:53:43,746 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1682996023643.5c823d6f15ad7b8fe82f9ae4beee6513. 2023-05-02 02:53:43,746 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1682996023643.5c823d6f15ad7b8fe82f9ae4beee6513. after waiting 0 ms 2023-05-02 02:53:43,746 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1682996023643.5c823d6f15ad7b8fe82f9ae4beee6513. 2023-05-02 02:53:43,746 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1682996023643.5c823d6f15ad7b8fe82f9ae4beee6513. 2023-05-02 02:53:43,746 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for 5c823d6f15ad7b8fe82f9ae4beee6513: 2023-05-02 02:53:43,751 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-05-02 02:53:43,766 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1682996023643.5c823d6f15ad7b8fe82f9ae4beee6513.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1682996023753"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682996023753"}]},"ts":"1682996023753"} 2023-05-02 02:53:43,796 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-05-02 02:53:43,798 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-05-02 02:53:43,803 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682996023798"}]},"ts":"1682996023798"} 2023-05-02 02:53:43,806 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-05-02 02:53:43,810 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-05-02 02:53:43,812 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-05-02 02:53:43,812 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-05-02 02:53:43,812 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-05-02 02:53:43,812 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-05-02 02:53:43,814 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=5c823d6f15ad7b8fe82f9ae4beee6513, ASSIGN}] 2023-05-02 02:53:43,818 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=5c823d6f15ad7b8fe82f9ae4beee6513, ASSIGN 2023-05-02 02:53:43,819 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=5c823d6f15ad7b8fe82f9ae4beee6513, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42019,1682996021676; forceNewPlan=false, retain=false 2023-05-02 02:53:43,971 INFO [jenkins-hbase4:41119] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-05-02 02:53:43,972 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=5c823d6f15ad7b8fe82f9ae4beee6513, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:43,973 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1682996023643.5c823d6f15ad7b8fe82f9ae4beee6513.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1682996023972"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1682996023972"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682996023972"}]},"ts":"1682996023972"} 2023-05-02 02:53:43,979 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 5c823d6f15ad7b8fe82f9ae4beee6513, server=jenkins-hbase4.apache.org,42019,1682996021676}] 2023-05-02 02:53:44,134 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:44,134 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-05-02 02:53:44,139 INFO [RS-EventLoopGroup-5-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58764, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-05-02 02:53:44,145 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1682996023643.5c823d6f15ad7b8fe82f9ae4beee6513. 2023-05-02 02:53:44,147 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 5c823d6f15ad7b8fe82f9ae4beee6513, NAME => 'hbase:namespace,,1682996023643.5c823d6f15ad7b8fe82f9ae4beee6513.', STARTKEY => '', ENDKEY => ''} 2023-05-02 02:53:44,148 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-02 02:53:44,148 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 5c823d6f15ad7b8fe82f9ae4beee6513 2023-05-02 02:53:44,149 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1682996023643.5c823d6f15ad7b8fe82f9ae4beee6513.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-02 02:53:44,149 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 5c823d6f15ad7b8fe82f9ae4beee6513 2023-05-02 02:53:44,149 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 5c823d6f15ad7b8fe82f9ae4beee6513 2023-05-02 02:53:44,151 INFO [StoreOpener-5c823d6f15ad7b8fe82f9ae4beee6513-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 5c823d6f15ad7b8fe82f9ae4beee6513 2023-05-02 02:53:44,153 DEBUG [StoreOpener-5c823d6f15ad7b8fe82f9ae4beee6513-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/hbase/namespace/5c823d6f15ad7b8fe82f9ae4beee6513/info 2023-05-02 02:53:44,153 DEBUG [StoreOpener-5c823d6f15ad7b8fe82f9ae4beee6513-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/hbase/namespace/5c823d6f15ad7b8fe82f9ae4beee6513/info 2023-05-02 02:53:44,154 INFO [StoreOpener-5c823d6f15ad7b8fe82f9ae4beee6513-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5c823d6f15ad7b8fe82f9ae4beee6513 columnFamilyName info 2023-05-02 02:53:44,155 INFO [StoreOpener-5c823d6f15ad7b8fe82f9ae4beee6513-1] regionserver.HStore(310): Store=5c823d6f15ad7b8fe82f9ae4beee6513/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-02 02:53:44,157 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/hbase/namespace/5c823d6f15ad7b8fe82f9ae4beee6513 2023-05-02 02:53:44,158 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/hbase/namespace/5c823d6f15ad7b8fe82f9ae4beee6513 2023-05-02 02:53:44,163 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 5c823d6f15ad7b8fe82f9ae4beee6513 2023-05-02 02:53:44,167 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/hbase/namespace/5c823d6f15ad7b8fe82f9ae4beee6513/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-05-02 02:53:44,167 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 5c823d6f15ad7b8fe82f9ae4beee6513; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66572821, jitterRate=-0.00798766314983368}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-05-02 02:53:44,167 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 5c823d6f15ad7b8fe82f9ae4beee6513: 2023-05-02 02:53:44,172 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:namespace,,1682996023643.5c823d6f15ad7b8fe82f9ae4beee6513., pid=6, masterSystemTime=1682996024134 2023-05-02 02:53:44,177 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:namespace,,1682996023643.5c823d6f15ad7b8fe82f9ae4beee6513. 2023-05-02 02:53:44,177 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1682996023643.5c823d6f15ad7b8fe82f9ae4beee6513. 2023-05-02 02:53:44,179 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=5c823d6f15ad7b8fe82f9ae4beee6513, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:44,179 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1682996023643.5c823d6f15ad7b8fe82f9ae4beee6513.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1682996024178"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1682996024178"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1682996024178"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682996024178"}]},"ts":"1682996024178"} 2023-05-02 02:53:44,186 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-05-02 02:53:44,186 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 5c823d6f15ad7b8fe82f9ae4beee6513, server=jenkins-hbase4.apache.org,42019,1682996021676 in 203 msec 2023-05-02 02:53:44,190 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-05-02 02:53:44,190 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=5c823d6f15ad7b8fe82f9ae4beee6513, ASSIGN in 372 msec 2023-05-02 02:53:44,191 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-05-02 02:53:44,192 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682996024192"}]},"ts":"1682996024192"} 2023-05-02 02:53:44,194 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-05-02 02:53:44,331 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-05-02 02:53:44,332 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-05-02 02:53:44,335 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 687 msec 2023-05-02 02:53:44,391 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-05-02 02:53:44,391 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-05-02 02:53:44,414 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-05-02 02:53:44,415 INFO [RS-EventLoopGroup-5-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58766, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-05-02 02:53:44,432 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-05-02 02:53:44,560 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-05-02 02:53:44,620 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 197 msec 2023-05-02 02:53:44,627 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-05-02 02:53:44,691 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-05-02 02:53:44,773 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 144 msec 2023-05-02 02:53:44,869 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-05-02 02:53:44,883 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-05-02 02:53:44,883 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 3.137sec 2023-05-02 02:53:44,887 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-05-02 02:53:44,889 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-05-02 02:53:44,889 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-05-02 02:53:44,891 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,41119,1682996020397-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-05-02 02:53:44,892 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,41119,1682996020397-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-05-02 02:53:44,903 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-05-02 02:53:44,911 DEBUG [Listener at localhost/37119] zookeeper.ReadOnlyZKClient(139): Connect 0x51c18fa6 to 127.0.0.1:53331 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-05-02 02:53:44,916 DEBUG [Listener at localhost/37119] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1f6a6811, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-05-02 02:53:44,930 DEBUG [hconnection-0x2c67b29-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-05-02 02:53:44,941 INFO [RS-EventLoopGroup-4-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:57932, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-05-02 02:53:44,951 INFO [Listener at localhost/37119] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,41119,1682996020397 2023-05-02 02:53:44,980 INFO [Listener at localhost/37119] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=358, OpenFileDescriptor=692, MaxFileDescriptor=60000, SystemLoadAverage=315, ProcessCount=170, AvailableMemoryMB=14048 2023-05-02 02:53:44,985 WARN [Listener at localhost/37119] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-02 02:53:44,993 DEBUG [Listener at localhost/37119] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-05-02 02:53:44,996 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:34000, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-05-02 02:53:45,010 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-05-02 02:53:45,010 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-05-02 02:53:45,011 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-05-02 02:53:45,016 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,019 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-05-02 02:53:45,021 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-05-02 02:53:45,024 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa 2023-05-02 02:53:45,024 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa empty. 2023-05-02 02:53:45,027 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa 2023-05-02 02:53:45,027 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-05-02 02:53:45,053 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionUnevenDaughter/.tabledesc/.tableinfo.0000000001 2023-05-02 02:53:45,055 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(7675): creating {ENCODED => 084b1607f7ed8615120a04e4729fbafa, NAME => 'testSplitTableRegionUnevenDaughter,,1682996025014.084b1607f7ed8615120a04e4729fbafa.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionUnevenDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp 2023-05-02 02:53:45,075 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegionUnevenDaughter,,1682996025014.084b1607f7ed8615120a04e4729fbafa.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-02 02:53:45,075 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1604): Closing 084b1607f7ed8615120a04e4729fbafa, disabling compactions & flushes 2023-05-02 02:53:45,075 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegionUnevenDaughter,,1682996025014.084b1607f7ed8615120a04e4729fbafa. 2023-05-02 02:53:45,075 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1682996025014.084b1607f7ed8615120a04e4729fbafa. 2023-05-02 02:53:45,075 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionUnevenDaughter,,1682996025014.084b1607f7ed8615120a04e4729fbafa. after waiting 0 ms 2023-05-02 02:53:45,076 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionUnevenDaughter,,1682996025014.084b1607f7ed8615120a04e4729fbafa. 2023-05-02 02:53:45,076 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegionUnevenDaughter,,1682996025014.084b1607f7ed8615120a04e4729fbafa. 2023-05-02 02:53:45,076 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1558): Region close journal for 084b1607f7ed8615120a04e4729fbafa: 2023-05-02 02:53:45,080 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-05-02 02:53:45,082 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1682996025014.084b1607f7ed8615120a04e4729fbafa.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1682996025081"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682996025081"}]},"ts":"1682996025081"} 2023-05-02 02:53:45,084 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-05-02 02:53:45,086 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-05-02 02:53:45,086 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682996025086"}]},"ts":"1682996025086"} 2023-05-02 02:53:45,088 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLING in hbase:meta 2023-05-02 02:53:45,092 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-05-02 02:53:45,093 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-05-02 02:53:45,094 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-05-02 02:53:45,094 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-05-02 02:53:45,094 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-05-02 02:53:45,094 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=084b1607f7ed8615120a04e4729fbafa, ASSIGN}] 2023-05-02 02:53:45,097 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=084b1607f7ed8615120a04e4729fbafa, ASSIGN 2023-05-02 02:53:45,098 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=084b1607f7ed8615120a04e4729fbafa, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,38925,1682996021633; forceNewPlan=false, retain=false 2023-05-02 02:53:45,249 INFO [jenkins-hbase4:41119] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-05-02 02:53:45,250 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=084b1607f7ed8615120a04e4729fbafa, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:45,250 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1682996025014.084b1607f7ed8615120a04e4729fbafa.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1682996025250"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1682996025250"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682996025250"}]},"ts":"1682996025250"} 2023-05-02 02:53:45,254 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 084b1607f7ed8615120a04e4729fbafa, server=jenkins-hbase4.apache.org,38925,1682996021633}] 2023-05-02 02:53:45,414 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionUnevenDaughter,,1682996025014.084b1607f7ed8615120a04e4729fbafa. 2023-05-02 02:53:45,414 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 084b1607f7ed8615120a04e4729fbafa, NAME => 'testSplitTableRegionUnevenDaughter,,1682996025014.084b1607f7ed8615120a04e4729fbafa.', STARTKEY => '', ENDKEY => ''} 2023-05-02 02:53:45,415 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-02 02:53:45,415 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter 084b1607f7ed8615120a04e4729fbafa 2023-05-02 02:53:45,415 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionUnevenDaughter,,1682996025014.084b1607f7ed8615120a04e4729fbafa.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-02 02:53:45,415 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 084b1607f7ed8615120a04e4729fbafa 2023-05-02 02:53:45,415 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 084b1607f7ed8615120a04e4729fbafa 2023-05-02 02:53:45,417 INFO [StoreOpener-084b1607f7ed8615120a04e4729fbafa-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 084b1607f7ed8615120a04e4729fbafa 2023-05-02 02:53:45,419 DEBUG [StoreOpener-084b1607f7ed8615120a04e4729fbafa-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1 2023-05-02 02:53:45,420 DEBUG [StoreOpener-084b1607f7ed8615120a04e4729fbafa-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1 2023-05-02 02:53:45,420 INFO [StoreOpener-084b1607f7ed8615120a04e4729fbafa-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 084b1607f7ed8615120a04e4729fbafa columnFamilyName cf1 2023-05-02 02:53:45,421 INFO [StoreOpener-084b1607f7ed8615120a04e4729fbafa-1] regionserver.HStore(310): Store=084b1607f7ed8615120a04e4729fbafa/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-02 02:53:45,421 INFO [StoreOpener-084b1607f7ed8615120a04e4729fbafa-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 084b1607f7ed8615120a04e4729fbafa 2023-05-02 02:53:45,423 DEBUG [StoreOpener-084b1607f7ed8615120a04e4729fbafa-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2 2023-05-02 02:53:45,423 DEBUG [StoreOpener-084b1607f7ed8615120a04e4729fbafa-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2 2023-05-02 02:53:45,424 INFO [StoreOpener-084b1607f7ed8615120a04e4729fbafa-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 084b1607f7ed8615120a04e4729fbafa columnFamilyName cf2 2023-05-02 02:53:45,425 INFO [StoreOpener-084b1607f7ed8615120a04e4729fbafa-1] regionserver.HStore(310): Store=084b1607f7ed8615120a04e4729fbafa/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-02 02:53:45,427 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa 2023-05-02 02:53:45,429 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa 2023-05-02 02:53:45,432 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-02 02:53:45,433 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 084b1607f7ed8615120a04e4729fbafa 2023-05-02 02:53:45,438 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-05-02 02:53:45,438 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 084b1607f7ed8615120a04e4729fbafa; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69729302, jitterRate=0.039047569036483765}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-02 02:53:45,439 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 084b1607f7ed8615120a04e4729fbafa: 2023-05-02 02:53:45,440 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1682996025014.084b1607f7ed8615120a04e4729fbafa., pid=11, masterSystemTime=1682996025407 2023-05-02 02:53:45,443 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1682996025014.084b1607f7ed8615120a04e4729fbafa. 2023-05-02 02:53:45,443 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionUnevenDaughter,,1682996025014.084b1607f7ed8615120a04e4729fbafa. 2023-05-02 02:53:45,444 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=084b1607f7ed8615120a04e4729fbafa, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:45,445 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1682996025014.084b1607f7ed8615120a04e4729fbafa.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1682996025444"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1682996025444"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1682996025444"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682996025444"}]},"ts":"1682996025444"} 2023-05-02 02:53:45,450 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-05-02 02:53:45,450 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 084b1607f7ed8615120a04e4729fbafa, server=jenkins-hbase4.apache.org,38925,1682996021633 in 194 msec 2023-05-02 02:53:45,454 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-05-02 02:53:45,454 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=084b1607f7ed8615120a04e4729fbafa, ASSIGN in 357 msec 2023-05-02 02:53:45,455 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-05-02 02:53:45,456 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682996025456"}]},"ts":"1682996025456"} 2023-05-02 02:53:45,461 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLED in hbase:meta 2023-05-02 02:53:45,465 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-05-02 02:53:45,468 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter in 451 msec 2023-05-02 02:53:45,541 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-02 02:53:45,548 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,549 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-05-02 02:53:45,549 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:45,549 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-05-02 02:53:45,549 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-05-02 02:53:45,550 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,550 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,551 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:45,551 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:45,551 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:45,551 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:45,551 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:45,551 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:45,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:45,551 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:45,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:45,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:45,552 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:45,552 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-02 02:53:45,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,553 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,553 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,553 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:53:45,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:53:45,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:53:45,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,553 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,553 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,554 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,555 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,555 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,555 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,556 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-02 02:53:45,556 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-02 02:53:45,556 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-02 02:53:45,556 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:45,556 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:45,556 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-02 02:53:45,557 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-02 02:53:45,556 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:45,557 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-02 02:53:45,557 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42019,1682996021676' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-02 02:53:45,557 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-02 02:53:45,557 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-02 02:53:45,557 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-02 02:53:45,558 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-02 02:53:45,558 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-02 02:53:45,558 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1682996025014.084b1607f7ed8615120a04e4729fbafa. 2023-05-02 02:53:45,558 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1682996025014.084b1607f7ed8615120a04e4729fbafa. started... 2023-05-02 02:53:45,559 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 084b1607f7ed8615120a04e4729fbafa 2/2 column families, dataSize=120 B heapSize=928 B 2023-05-02 02:53:45,560 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,560 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,560 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,560 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:53:45,561 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,561 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:53:45,634 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/.tmp/cf1/3663054878d64ef28b37a569207194d8 2023-05-02 02:53:45,697 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/.tmp/cf2/b5f2f094012f478e9aa22a132932427f 2023-05-02 02:53:45,707 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/.tmp/cf1/3663054878d64ef28b37a569207194d8 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1/3663054878d64ef28b37a569207194d8 2023-05-02 02:53:45,716 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1/3663054878d64ef28b37a569207194d8, entries=2, sequenceid=6, filesize=4.8 K 2023-05-02 02:53:45,719 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/.tmp/cf2/b5f2f094012f478e9aa22a132932427f as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2/b5f2f094012f478e9aa22a132932427f 2023-05-02 02:53:45,727 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2/b5f2f094012f478e9aa22a132932427f, entries=2, sequenceid=6, filesize=4.8 K 2023-05-02 02:53:45,729 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 084b1607f7ed8615120a04e4729fbafa in 170ms, sequenceid=6, compaction requested=false 2023-05-02 02:53:45,731 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionUnevenDaughter' 2023-05-02 02:53:45,733 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 084b1607f7ed8615120a04e4729fbafa: 2023-05-02 02:53:45,733 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1682996025014.084b1607f7ed8615120a04e4729fbafa. 2023-05-02 02:53:45,734 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-02 02:53:45,734 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-02 02:53:45,734 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:45,734 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-02 02:53:45,734 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38925,1682996021633' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-02 02:53:45,737 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:45,737 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:45,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:53:45,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:53:45,737 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,737 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:53:45,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:53:45,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:53:45,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:45,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:45,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:45,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:53:45,741 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38925,1682996021633' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-05-02 02:53:45,741 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-05-02 02:53:45,741 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5733a331[Count = 0] remaining members to acquire global barrier 2023-05-02 02:53:45,741 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,746 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,746 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,746 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,746 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,746 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,746 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-02 02:53:45,746 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:45,746 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-02 02:53:45,746 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,746 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-02 02:53:45,746 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-02 02:53:45,746 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-02 02:53:45,746 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,38925,1682996021633' in zk 2023-05-02 02:53:45,746 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-02 02:53:45,746 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,42019,1682996021676' in zk 2023-05-02 02:53:45,746 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-02 02:53:45,747 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,39081,1682996021579' in zk 2023-05-02 02:53:45,750 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:45,750 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-02 02:53:45,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:45,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:53:45,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:53:45,750 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:45,750 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-02 02:53:45,751 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-02 02:53:45,752 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:45,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:53:45,752 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-02 02:53:45,752 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-02 02:53:45,752 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:45,752 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-02 02:53:45,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:53:45,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:45,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:45,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:45,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:53:45,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:45,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:45,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:45,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,38925,1682996021633': 2023-05-02 02:53:45,757 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-05-02 02:53:45,758 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-02 02:53:45,757 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38925,1682996021633' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-05-02 02:53:45,758 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-02 02:53:45,758 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,758 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-02 02:53:45,760 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,760 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:45,760 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,760 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,760 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,760 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:45,760 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,760 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,760 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:53:45,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,760 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:45,761 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:45,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:53:45,761 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:45,761 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:45,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:45,761 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:45,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:45,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:45,762 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:45,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:53:45,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,763 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:45,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:53:45,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:45,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:45,766 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:45,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:45,766 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:45,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:53:45,767 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:45,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:45,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:45,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:45,779 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:45,779 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,779 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:45,779 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:45,779 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:45,779 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:45,779 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:45,780 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:45,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:45,780 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:45,779 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:45,779 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:45,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:45,780 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:45,780 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-05-02 02:53:45,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:45,780 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-02 02:53:45,780 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,780 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:45,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:45,780 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:45,781 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:45,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:45,781 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,781 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:45,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:45,781 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:45,781 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:45,781 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,781 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,782 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:45,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:45,783 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-05-02 02:53:45,783 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-02 02:53:45,883 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-02 02:53:45,888 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-02 02:53:45,912 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-02 02:53:45,915 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-05-02 02:53:45,915 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,915 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-05-02 02:53:45,915 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:45,916 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-05-02 02:53:45,916 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-05-02 02:53:45,917 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,917 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,918 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:45,918 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:45,918 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:45,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:45,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:45,918 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:45,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:45,919 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:45,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:45,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:45,919 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:45,919 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-02 02:53:45,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,920 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,920 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,920 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:53:45,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:53:45,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:53:45,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,920 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,920 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,921 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,921 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-05-02 02:53:45,921 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-05-02 02:53:45,921 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,921 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,921 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-02 02:53:45,921 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-05-02 02:53:45,921 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:45,921 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,922 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-02 02:53:45,922 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-02 02:53:45,922 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:45,922 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-02 02:53:45,922 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-02 02:53:45,922 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:45,922 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-02 02:53:45,923 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-02 02:53:45,923 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-02 02:53:45,923 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-02 02:53:45,923 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42019,1682996021676' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-02 02:53:45,923 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-02 02:53:45,923 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-02 02:53:45,923 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1682996025014.084b1607f7ed8615120a04e4729fbafa. 2023-05-02 02:53:45,923 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1682996025014.084b1607f7ed8615120a04e4729fbafa. started... 2023-05-02 02:53:45,924 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 084b1607f7ed8615120a04e4729fbafa 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-02 02:53:45,926 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,926 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,927 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,927 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:45,927 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:53:45,927 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:53:45,946 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/.tmp/cf1/2cb812bd81a94565a0a1cf59a289bd05 2023-05-02 02:53:45,979 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/.tmp/cf2/ea424b3182454ff5a1a8d6c3795602a2 2023-05-02 02:53:45,989 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/.tmp/cf1/2cb812bd81a94565a0a1cf59a289bd05 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1/2cb812bd81a94565a0a1cf59a289bd05 2023-05-02 02:53:45,999 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1/2cb812bd81a94565a0a1cf59a289bd05, entries=10, sequenceid=19, filesize=5.0 K 2023-05-02 02:53:46,000 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/.tmp/cf2/ea424b3182454ff5a1a8d6c3795602a2 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2/ea424b3182454ff5a1a8d6c3795602a2 2023-05-02 02:53:46,008 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2/ea424b3182454ff5a1a8d6c3795602a2, entries=10, sequenceid=19, filesize=5.0 K 2023-05-02 02:53:46,010 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 084b1607f7ed8615120a04e4729fbafa in 85ms, sequenceid=19, compaction requested=false 2023-05-02 02:53:46,010 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 084b1607f7ed8615120a04e4729fbafa: 2023-05-02 02:53:46,010 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1682996025014.084b1607f7ed8615120a04e4729fbafa. 2023-05-02 02:53:46,010 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-02 02:53:46,010 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-02 02:53:46,010 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:46,011 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-02 02:53:46,011 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38925,1682996021633' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-02 02:53:46,012 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:46,012 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:46,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:53:46,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:53:46,013 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,013 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:53:46,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:53:46,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:53:46,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:46,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:46,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:46,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:53:46,016 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38925,1682996021633' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-05-02 02:53:46,016 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7d229f88[Count = 0] remaining members to acquire global barrier 2023-05-02 02:53:46,016 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-05-02 02:53:46,016 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,017 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,017 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,017 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,017 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,017 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,017 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,017 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-02 02:53:46,017 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:46,017 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-02 02:53:46,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,018 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-02 02:53:46,018 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-02 02:53:46,018 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-02 02:53:46,018 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-02 02:53:46,018 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-02 02:53:46,018 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,39081,1682996021579' in zk 2023-05-02 02:53:46,018 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,42019,1682996021676' in zk 2023-05-02 02:53:46,018 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,38925,1682996021633' in zk 2023-05-02 02:53:46,020 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-02 02:53:46,020 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:46,020 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:46,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:46,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:53:46,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:53:46,020 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-02 02:53:46,022 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-02 02:53:46,022 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:46,022 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-02 02:53:46,022 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-02 02:53:46,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:53:46,023 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:46,023 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-02 02:53:46,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:53:46,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:46,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:46,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:46,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:53:46,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:46,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:46,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:46,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,38925,1682996021633': 2023-05-02 02:53:46,029 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38925,1682996021633' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-05-02 02:53:46,029 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-05-02 02:53:46,029 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-02 02:53:46,029 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-02 02:53:46,029 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,029 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-02 02:53:46,031 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,031 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,031 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,031 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,031 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:46,031 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,031 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:46,031 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,031 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:46,031 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:53:46,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:53:46,032 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:46,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:46,032 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:46,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:46,032 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:46,032 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:46,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:53:46,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:46,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,033 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:46,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,034 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:46,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:53:46,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:46,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:46,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:46,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:53:46,037 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:46,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,038 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:46,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:46,038 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:46,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:46,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:46,048 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:46,048 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:46,048 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:46,048 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:46,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:46,049 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:46,048 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,049 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:46,049 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:46,049 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,049 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:46,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:46,049 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:46,048 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:46,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:46,049 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:46,049 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:46,050 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:46,050 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,050 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:46,050 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:46,050 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:46,050 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:46,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:46,050 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:46,050 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-05-02 02:53:46,051 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-02 02:53:46,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:46,051 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:46,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:46,051 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,052 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-05-02 02:53:46,052 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-02 02:53:46,052 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,152 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-02 02:53:46,153 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-02 02:53:46,178 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-02 02:53:46,181 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-05-02 02:53:46,181 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,181 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-05-02 02:53:46,181 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:46,182 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-05-02 02:53:46,182 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-05-02 02:53:46,183 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,183 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,185 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:46,185 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:46,185 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:46,185 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:46,185 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:46,185 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:46,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:46,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:46,185 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:46,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:46,185 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:46,186 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-02 02:53:46,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,186 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,186 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,186 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:53:46,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,187 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:53:46,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,187 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:53:46,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,187 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,187 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-05-02 02:53:46,187 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-05-02 02:53:46,187 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-05-02 02:53:46,189 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,190 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,190 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-02 02:53:46,190 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-02 02:53:46,187 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,190 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:46,190 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-02 02:53:46,190 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-02 02:53:46,190 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-02 02:53:46,190 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:46,190 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-02 02:53:46,190 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:46,191 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-02 02:53:46,191 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-02 02:53:46,191 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-02 02:53:46,191 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42019,1682996021676' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-02 02:53:46,192 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-02 02:53:46,192 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-02 02:53:46,192 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1682996025014.084b1607f7ed8615120a04e4729fbafa. 2023-05-02 02:53:46,192 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1682996025014.084b1607f7ed8615120a04e4729fbafa. started... 2023-05-02 02:53:46,193 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 084b1607f7ed8615120a04e4729fbafa 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-02 02:53:46,193 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,194 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,194 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,194 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:53:46,195 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,195 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:53:46,215 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/.tmp/cf1/7285a69c46ad4b738dfaa96fd16f3ad9 2023-05-02 02:53:46,245 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/.tmp/cf2/9cf56291619249dda5dc07d66464b1af 2023-05-02 02:53:46,254 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/.tmp/cf1/7285a69c46ad4b738dfaa96fd16f3ad9 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1/7285a69c46ad4b738dfaa96fd16f3ad9 2023-05-02 02:53:46,262 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1/7285a69c46ad4b738dfaa96fd16f3ad9, entries=10, sequenceid=32, filesize=5.0 K 2023-05-02 02:53:46,263 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/.tmp/cf2/9cf56291619249dda5dc07d66464b1af as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2/9cf56291619249dda5dc07d66464b1af 2023-05-02 02:53:46,271 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2/9cf56291619249dda5dc07d66464b1af, entries=10, sequenceid=32, filesize=5.0 K 2023-05-02 02:53:46,272 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 084b1607f7ed8615120a04e4729fbafa in 80ms, sequenceid=32, compaction requested=true 2023-05-02 02:53:46,272 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 084b1607f7ed8615120a04e4729fbafa: 2023-05-02 02:53:46,272 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1682996025014.084b1607f7ed8615120a04e4729fbafa. 2023-05-02 02:53:46,272 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-02 02:53:46,273 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-02 02:53:46,273 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:46,273 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-02 02:53:46,273 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38925,1682996021633' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-02 02:53:46,275 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,275 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:46,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:46,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:53:46,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:53:46,275 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,275 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:53:46,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:53:46,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:53:46,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:46,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:46,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:46,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:53:46,278 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38925,1682996021633' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-05-02 02:53:46,278 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3d6ac80d[Count = 0] remaining members to acquire global barrier 2023-05-02 02:53:46,278 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-05-02 02:53:46,278 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,280 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,280 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,280 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,280 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,281 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,280 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,281 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-02 02:53:46,281 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-02 02:53:46,281 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-02 02:53:46,281 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-02 02:53:46,281 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,42019,1682996021676' in zk 2023-05-02 02:53:46,281 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:46,281 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-02 02:53:46,281 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-02 02:53:46,281 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-02 02:53:46,281 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,39081,1682996021579' in zk 2023-05-02 02:53:46,281 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,38925,1682996021633' in zk 2023-05-02 02:53:46,284 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-02 02:53:46,284 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:46,284 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-02 02:53:46,285 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-02 02:53:46,285 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-02 02:53:46,285 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:46,285 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:46,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:46,285 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:46,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:53:46,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:53:46,285 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-02 02:53:46,287 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-02 02:53:46,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:53:46,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:53:46,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:46,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:46,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:46,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:53:46,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:46,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:46,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:46,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,38925,1682996021633': 2023-05-02 02:53:46,291 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38925,1682996021633' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-05-02 02:53:46,291 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-05-02 02:53:46,291 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-02 02:53:46,291 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-02 02:53:46,291 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,291 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-02 02:53:46,293 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,293 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,293 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,293 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:53:46,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:53:46,293 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,293 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:46,293 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,293 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:46,294 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,294 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:46,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,294 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:46,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:46,294 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:46,294 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:46,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,294 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:46,294 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:46,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:46,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:46,295 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:46,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,298 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:46,298 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:46,299 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:46,304 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:46,304 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,304 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:46,304 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:46,304 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:46,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:46,304 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:46,304 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:46,304 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:46,304 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:46,304 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:46,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:46,304 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:46,304 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:46,304 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:46,305 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-05-02 02:53:46,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:46,304 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:46,305 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-02 02:53:46,304 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:46,305 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:46,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:46,305 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,305 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:46,305 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:46,305 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-05-02 02:53:46,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:46,305 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-02 02:53:46,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:46,305 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,306 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:46,306 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:46,306 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,306 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:53:46,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:53:46,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:53:46,309 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter because node does not exist (not an error) 2023-05-02 02:53:46,406 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-02 02:53:46,407 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-02 02:53:46,431 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-02 02:53:46,434 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-05-02 02:53:46,434 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,434 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-05-02 02:53:46,434 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:46,435 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-05-02 02:53:46,435 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-05-02 02:53:46,436 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,436 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,437 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:46,437 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:46,437 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:46,437 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:46,437 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:46,437 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:46,437 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:46,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:46,437 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:46,437 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-02 02:53:46,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:46,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:46,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,438 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,438 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,438 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:53:46,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:53:46,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,439 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:53:46,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,439 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,439 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,439 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-05-02 02:53:46,439 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-05-02 02:53:46,439 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,439 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-05-02 02:53:46,439 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,440 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,440 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-02 02:53:46,440 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-02 02:53:46,440 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:46,439 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-02 02:53:46,441 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-02 02:53:46,440 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:46,441 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-02 02:53:46,441 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-02 02:53:46,442 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-02 02:53:46,442 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42019,1682996021676' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-02 02:53:46,441 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:46,441 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-02 02:53:46,442 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-02 02:53:46,442 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-02 02:53:46,443 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-02 02:53:46,443 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1682996025014.084b1607f7ed8615120a04e4729fbafa. 2023-05-02 02:53:46,443 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1682996025014.084b1607f7ed8615120a04e4729fbafa. started... 2023-05-02 02:53:46,443 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 084b1607f7ed8615120a04e4729fbafa 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-02 02:53:46,444 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,444 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,445 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,445 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:53:46,445 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,445 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:53:46,463 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/.tmp/cf1/0a58882c16bc41b788b4deccd233becc 2023-05-02 02:53:46,495 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/.tmp/cf2/cf84b8983146452aa1df04c6b4dfbbb6 2023-05-02 02:53:46,505 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/.tmp/cf1/0a58882c16bc41b788b4deccd233becc as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1/0a58882c16bc41b788b4deccd233becc 2023-05-02 02:53:46,514 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1/0a58882c16bc41b788b4deccd233becc, entries=10, sequenceid=45, filesize=5.0 K 2023-05-02 02:53:46,516 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/.tmp/cf2/cf84b8983146452aa1df04c6b4dfbbb6 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2/cf84b8983146452aa1df04c6b4dfbbb6 2023-05-02 02:53:46,525 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2/cf84b8983146452aa1df04c6b4dfbbb6, entries=10, sequenceid=45, filesize=5.0 K 2023-05-02 02:53:46,527 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 084b1607f7ed8615120a04e4729fbafa in 84ms, sequenceid=45, compaction requested=true 2023-05-02 02:53:46,527 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 084b1607f7ed8615120a04e4729fbafa: 2023-05-02 02:53:46,527 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1682996025014.084b1607f7ed8615120a04e4729fbafa. 2023-05-02 02:53:46,527 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-02 02:53:46,528 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-02 02:53:46,528 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:46,528 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-02 02:53:46,528 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38925,1682996021633' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-02 02:53:46,536 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,536 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:46,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:46,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:53:46,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:53:46,537 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,537 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:53:46,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:53:46,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:53:46,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:46,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:46,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:46,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:53:46,539 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38925,1682996021633' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-05-02 02:53:46,539 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-05-02 02:53:46,539 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4b364110[Count = 0] remaining members to acquire global barrier 2023-05-02 02:53:46,540 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,541 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,541 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,541 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,541 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,541 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,541 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-02 02:53:46,541 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-02 02:53:46,541 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-02 02:53:46,541 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,541 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:46,541 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-02 02:53:46,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,541 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,39081,1682996021579' in zk 2023-05-02 02:53:46,541 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-02 02:53:46,542 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-02 02:53:46,541 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-02 02:53:46,542 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,38925,1682996021633' in zk 2023-05-02 02:53:46,542 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,42019,1682996021676' in zk 2023-05-02 02:53:46,543 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-02 02:53:46,543 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:46,543 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-02 02:53:46,544 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-02 02:53:46,544 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-02 02:53:46,544 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:46,544 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:46,544 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:46,545 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-02 02:53:46,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:46,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:53:46,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:53:46,544 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-02 02:53:46,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:53:46,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:53:46,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:46,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:46,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:46,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:53:46,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:46,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:46,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:46,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,38925,1682996021633': 2023-05-02 02:53:46,550 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38925,1682996021633' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-05-02 02:53:46,550 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-05-02 02:53:46,550 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-02 02:53:46,550 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-02 02:53:46,550 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,550 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-02 02:53:46,553 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,553 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,553 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,553 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,553 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,553 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:53:46,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:53:46,553 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:46,553 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,553 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:46,553 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:46,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,553 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:46,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:53:46,554 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:46,554 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:46,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:46,554 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:46,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:46,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:46,554 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:46,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,555 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:46,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:53:46,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:46,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:46,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:46,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:53:46,557 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:46,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,557 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:46,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:46,557 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:46,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:46,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:46,560 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:46,560 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,560 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:46,560 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:46,560 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:46,561 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:46,561 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:46,561 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-05-02 02:53:46,561 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-02 02:53:46,561 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:46,561 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:46,560 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:46,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:46,560 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:46,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:46,561 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,561 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:46,561 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-05-02 02:53:46,561 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:46,562 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-02 02:53:46,561 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:46,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:46,561 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:46,562 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,562 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:46,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:46,561 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:46,562 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:46,563 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:46,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:46,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:46,563 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:46,563 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,563 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,662 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-02 02:53:46,664 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-02 02:53:46,685 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-02 02:53:46,688 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-05-02 02:53:46,688 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,688 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-05-02 02:53:46,688 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:46,689 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-05-02 02:53:46,689 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-05-02 02:53:46,690 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,690 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,691 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:46,691 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:46,691 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:46,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:46,691 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:46,691 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:46,691 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:46,691 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:46,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:46,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:46,691 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:46,692 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-02 02:53:46,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,692 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,692 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,692 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:53:46,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:53:46,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,693 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,693 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:53:46,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,693 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,693 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-05-02 02:53:46,693 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,693 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-02 02:53:46,693 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-05-02 02:53:46,693 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:46,693 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,694 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-02 02:53:46,694 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-05-02 02:53:46,694 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-02 02:53:46,694 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,694 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:46,694 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-02 02:53:46,694 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-02 02:53:46,696 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-02 02:53:46,696 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-02 02:53:46,696 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-02 02:53:46,696 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-02 02:53:46,696 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:46,696 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1682996025014.084b1607f7ed8615120a04e4729fbafa. 2023-05-02 02:53:46,698 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-02 02:53:46,698 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1682996025014.084b1607f7ed8615120a04e4729fbafa. started... 2023-05-02 02:53:46,699 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-02 02:53:46,699 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42019,1682996021676' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-02 02:53:46,699 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 084b1607f7ed8615120a04e4729fbafa 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-02 02:53:46,700 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,702 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,702 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,702 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:53:46,703 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,703 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:53:46,720 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/.tmp/cf1/1893934900b2482ea071cead41006e9d 2023-05-02 02:53:46,755 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/.tmp/cf2/8d86bdc24ade4df9963a5d3f077111d2 2023-05-02 02:53:46,764 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/.tmp/cf1/1893934900b2482ea071cead41006e9d as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1/1893934900b2482ea071cead41006e9d 2023-05-02 02:53:46,771 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1/1893934900b2482ea071cead41006e9d, entries=10, sequenceid=58, filesize=5.0 K 2023-05-02 02:53:46,772 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/.tmp/cf2/8d86bdc24ade4df9963a5d3f077111d2 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2/8d86bdc24ade4df9963a5d3f077111d2 2023-05-02 02:53:46,780 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2/8d86bdc24ade4df9963a5d3f077111d2, entries=10, sequenceid=58, filesize=5.0 K 2023-05-02 02:53:46,781 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 084b1607f7ed8615120a04e4729fbafa in 82ms, sequenceid=58, compaction requested=true 2023-05-02 02:53:46,781 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 084b1607f7ed8615120a04e4729fbafa: 2023-05-02 02:53:46,781 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1682996025014.084b1607f7ed8615120a04e4729fbafa. 2023-05-02 02:53:46,781 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-02 02:53:46,783 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-02 02:53:46,783 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:46,783 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-02 02:53:46,783 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38925,1682996021633' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-02 02:53:46,785 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,785 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:46,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:46,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:53:46,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:53:46,786 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,786 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:53:46,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:53:46,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:53:46,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:46,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:46,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:46,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:53:46,789 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38925,1682996021633' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-05-02 02:53:46,789 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@156496c[Count = 0] remaining members to acquire global barrier 2023-05-02 02:53:46,789 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-05-02 02:53:46,789 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,791 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,791 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,791 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,791 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,791 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,791 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,791 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:46,791 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-02 02:53:46,791 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-02 02:53:46,791 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-02 02:53:46,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,791 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-02 02:53:46,791 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,42019,1682996021676' in zk 2023-05-02 02:53:46,791 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-02 02:53:46,792 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,39081,1682996021579' in zk 2023-05-02 02:53:46,792 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-02 02:53:46,792 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-02 02:53:46,792 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,38925,1682996021633' in zk 2023-05-02 02:53:46,793 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-02 02:53:46,793 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:46,793 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-02 02:53:46,798 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-02 02:53:46,798 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:46,798 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:46,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:46,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:53:46,798 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-02 02:53:46,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:53:46,798 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-02 02:53:46,798 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:46,799 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-02 02:53:46,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:53:46,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:53:46,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:46,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:46,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:46,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:53:46,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:46,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:46,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:46,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,38925,1682996021633': 2023-05-02 02:53:46,803 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38925,1682996021633' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-05-02 02:53:46,803 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-05-02 02:53:46,803 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-02 02:53:46,803 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-02 02:53:46,803 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,803 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-02 02:53:46,805 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,805 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:46,805 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,805 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,805 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,805 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,805 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,805 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:46,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,805 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:46,805 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,805 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:46,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:46,806 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:46,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:53:46,806 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:46,806 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:46,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:46,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:53:46,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,806 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:46,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:46,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:53:46,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,807 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:46,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:53:46,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:46,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:46,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:46,809 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:46,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:53:46,809 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:46,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,809 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:46,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:46,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:46,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:46,814 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:46,815 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,815 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:46,815 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:46,815 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:46,815 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:46,815 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:46,815 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:46,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:46,815 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:46,815 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:46,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:46,815 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:46,815 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:46,815 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-05-02 02:53:46,815 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:46,815 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:46,815 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:46,815 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,816 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:46,816 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:46,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:46,816 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:46,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:46,815 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-02 02:53:46,816 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:46,817 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:46,817 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,817 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,817 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:46,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:46,817 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-05-02 02:53:46,817 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-02 02:53:46,917 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-02 02:53:46,919 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-02 02:53:46,942 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-02 02:53:46,944 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-05-02 02:53:46,944 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,945 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-05-02 02:53:46,945 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:46,945 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-05-02 02:53:46,946 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-05-02 02:53:46,946 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,946 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,948 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:46,948 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:46,948 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:46,948 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:46,948 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:46,948 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:46,948 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:46,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:46,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:46,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:46,949 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:46,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,949 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-02 02:53:46,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,950 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,950 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,950 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:53:46,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:53:46,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,950 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,950 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:53:46,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,951 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,951 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-05-02 02:53:46,951 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-05-02 02:53:46,951 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,951 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,951 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-02 02:53:46,951 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-02 02:53:46,951 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-05-02 02:53:46,951 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:46,951 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,952 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-02 02:53:46,952 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-02 02:53:46,952 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:46,951 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:46,953 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-02 02:53:46,952 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-02 02:53:46,953 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-02 02:53:46,953 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-02 02:53:46,953 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-02 02:53:46,953 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42019,1682996021676' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-02 02:53:46,953 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-02 02:53:46,953 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-02 02:53:46,953 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1682996025014.084b1607f7ed8615120a04e4729fbafa. 2023-05-02 02:53:46,954 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1682996025014.084b1607f7ed8615120a04e4729fbafa. started... 2023-05-02 02:53:46,954 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 084b1607f7ed8615120a04e4729fbafa 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-02 02:53:46,955 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,955 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,956 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,956 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:53:46,956 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:46,956 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:53:46,975 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/.tmp/cf1/0cbb9fddcfd74f8c853f11e6ad27b8c1 2023-05-02 02:53:46,999 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/.tmp/cf2/7fcf6b93d24a4fb5bb305342e41fd972 2023-05-02 02:53:47,008 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/.tmp/cf1/0cbb9fddcfd74f8c853f11e6ad27b8c1 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1/0cbb9fddcfd74f8c853f11e6ad27b8c1 2023-05-02 02:53:47,015 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1/0cbb9fddcfd74f8c853f11e6ad27b8c1, entries=10, sequenceid=71, filesize=5.0 K 2023-05-02 02:53:47,016 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/.tmp/cf2/7fcf6b93d24a4fb5bb305342e41fd972 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2/7fcf6b93d24a4fb5bb305342e41fd972 2023-05-02 02:53:47,023 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2/7fcf6b93d24a4fb5bb305342e41fd972, entries=10, sequenceid=71, filesize=5.0 K 2023-05-02 02:53:47,024 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 084b1607f7ed8615120a04e4729fbafa in 70ms, sequenceid=71, compaction requested=true 2023-05-02 02:53:47,025 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 084b1607f7ed8615120a04e4729fbafa: 2023-05-02 02:53:47,025 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1682996025014.084b1607f7ed8615120a04e4729fbafa. 2023-05-02 02:53:47,025 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-02 02:53:47,026 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-02 02:53:47,026 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:47,026 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-02 02:53:47,026 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38925,1682996021633' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-02 02:53:47,027 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:47,027 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:47,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:47,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:53:47,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:53:47,028 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:47,028 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:53:47,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:53:47,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:53:47,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-02 02:53:47,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:47,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:47,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:47,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:53:47,030 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38925,1682996021633' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-05-02 02:53:47,030 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@68614811[Count = 0] remaining members to acquire global barrier 2023-05-02 02:53:47,030 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-05-02 02:53:47,030 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:47,031 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:47,031 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:47,031 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:47,031 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:47,031 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:47,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:47,032 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:47,032 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-02 02:53:47,032 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:47,032 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-02 02:53:47,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:47,032 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-02 02:53:47,032 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-02 02:53:47,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:47,032 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-02 02:53:47,032 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-02 02:53:47,032 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-02 02:53:47,032 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,39081,1682996021579' in zk 2023-05-02 02:53:47,032 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,42019,1682996021676' in zk 2023-05-02 02:53:47,032 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,38925,1682996021633' in zk 2023-05-02 02:53:47,034 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-02 02:53:47,034 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:47,034 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-02 02:53:47,036 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-02 02:53:47,036 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:47,036 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:47,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:47,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:53:47,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:53:47,036 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-02 02:53:47,036 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-02 02:53:47,037 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:47,037 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-02 02:53:47,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:53:47,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:53:47,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-02 02:53:47,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:47,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:47,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:47,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:53:47,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-02 02:53:47,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:47,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:47,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:47,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,38925,1682996021633': 2023-05-02 02:53:47,041 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38925,1682996021633' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-05-02 02:53:47,041 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-05-02 02:53:47,041 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-02 02:53:47,041 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-02 02:53:47,041 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-05-02 02:53:47,041 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-02 02:53:47,042 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:47,042 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:47,043 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:47,043 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:47,042 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:47,042 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:47,042 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:47,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:47,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:47,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:53:47,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:47,043 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:47,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:53:47,043 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:47,043 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:47,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:47,043 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:47,043 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:47,043 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:47,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:47,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:47,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:47,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:53:47,044 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:47,044 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:47,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:47,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:47,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-02 02:53:47,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:47,044 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:47,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:53:47,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-02 02:53:47,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:47,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:47,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:47,046 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:47,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:53:47,047 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:47,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-02 02:53:47,047 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:47,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:47,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:47,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:47,050 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:47,050 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:47,050 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:47,050 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:47,050 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:47,050 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:47,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:47,050 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:47,050 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:47,050 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:47,050 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:47,050 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:47,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:47,051 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:47,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:47,051 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:47,050 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-05-02 02:53:47,051 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-02 02:53:47,050 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:47,051 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:47,051 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:47,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:47,051 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:47,051 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:47,052 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-05-02 02:53:47,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:47,052 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-02 02:53:47,052 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:47,052 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:47,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:47,052 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:47,052 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:47,053 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:47,053 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-02 02:53:47,152 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-02 02:53:47,154 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-02 02:53:47,177 DEBUG [Listener at localhost/37119] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:47,183 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=084b1607f7ed8615120a04e4729fbafa, daughterA=ea7683cd3436a62eb16418a3c875d563, daughterB=408e891cab9b919f4038b233ba34779d 2023-05-02 02:53:47,185 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=084b1607f7ed8615120a04e4729fbafa, daughterA=ea7683cd3436a62eb16418a3c875d563, daughterB=408e891cab9b919f4038b233ba34779d 2023-05-02 02:53:47,185 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=084b1607f7ed8615120a04e4729fbafa, daughterA=ea7683cd3436a62eb16418a3c875d563, daughterB=408e891cab9b919f4038b233ba34779d 2023-05-02 02:53:47,185 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=084b1607f7ed8615120a04e4729fbafa, daughterA=ea7683cd3436a62eb16418a3c875d563, daughterB=408e891cab9b919f4038b233ba34779d 2023-05-02 02:53:47,193 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=084b1607f7ed8615120a04e4729fbafa, UNASSIGN}] 2023-05-02 02:53:47,194 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=084b1607f7ed8615120a04e4729fbafa, UNASSIGN 2023-05-02 02:53:47,195 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=084b1607f7ed8615120a04e4729fbafa, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:47,195 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1682996025014.084b1607f7ed8615120a04e4729fbafa.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1682996027195"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1682996027195"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682996027195"}]},"ts":"1682996027195"} 2023-05-02 02:53:47,198 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; CloseRegionProcedure 084b1607f7ed8615120a04e4729fbafa, server=jenkins-hbase4.apache.org,38925,1682996021633}] 2023-05-02 02:53:47,358 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 084b1607f7ed8615120a04e4729fbafa 2023-05-02 02:53:47,358 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 084b1607f7ed8615120a04e4729fbafa, disabling compactions & flushes 2023-05-02 02:53:47,358 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionUnevenDaughter,,1682996025014.084b1607f7ed8615120a04e4729fbafa. 2023-05-02 02:53:47,358 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1682996025014.084b1607f7ed8615120a04e4729fbafa. 2023-05-02 02:53:47,358 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionUnevenDaughter,,1682996025014.084b1607f7ed8615120a04e4729fbafa. after waiting 0 ms 2023-05-02 02:53:47,358 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionUnevenDaughter,,1682996025014.084b1607f7ed8615120a04e4729fbafa. 2023-05-02 02:53:47,359 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 084b1607f7ed8615120a04e4729fbafa 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-05-02 02:53:47,379 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/.tmp/cf1/6d14eb71c6f244cba148da66de9514b6 2023-05-02 02:53:47,404 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/.tmp/cf2/a68c693682044c6c8576e930cd4eaf62 2023-05-02 02:53:47,412 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/.tmp/cf1/6d14eb71c6f244cba148da66de9514b6 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1/6d14eb71c6f244cba148da66de9514b6 2023-05-02 02:53:47,421 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1/6d14eb71c6f244cba148da66de9514b6, entries=8, sequenceid=82, filesize=5.0 K 2023-05-02 02:53:47,423 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/.tmp/cf2/a68c693682044c6c8576e930cd4eaf62 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2/a68c693682044c6c8576e930cd4eaf62 2023-05-02 02:53:47,432 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2/a68c693682044c6c8576e930cd4eaf62, entries=8, sequenceid=82, filesize=5.0 K 2023-05-02 02:53:47,434 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 084b1607f7ed8615120a04e4729fbafa in 76ms, sequenceid=82, compaction requested=true 2023-05-02 02:53:47,456 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-05-02 02:53:47,457 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-02 02:53:47,458 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionUnevenDaughter,,1682996025014.084b1607f7ed8615120a04e4729fbafa. 2023-05-02 02:53:47,458 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 084b1607f7ed8615120a04e4729fbafa: 2023-05-02 02:53:47,460 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 084b1607f7ed8615120a04e4729fbafa 2023-05-02 02:53:47,461 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=084b1607f7ed8615120a04e4729fbafa, regionState=CLOSED 2023-05-02 02:53:47,462 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1682996025014.084b1607f7ed8615120a04e4729fbafa.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1682996027461"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682996027461"}]},"ts":"1682996027461"} 2023-05-02 02:53:47,467 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-05-02 02:53:47,467 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; CloseRegionProcedure 084b1607f7ed8615120a04e4729fbafa, server=jenkins-hbase4.apache.org,38925,1682996021633 in 267 msec 2023-05-02 02:53:47,469 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=13, resume processing ppid=12 2023-05-02 02:53:47,470 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=084b1607f7ed8615120a04e4729fbafa, UNASSIGN in 274 msec 2023-05-02 02:53:47,489 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=12 splitting 14 storefiles, region=084b1607f7ed8615120a04e4729fbafa, threads=14 2023-05-02 02:53:47,490 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2/7fcf6b93d24a4fb5bb305342e41fd972 for region: 084b1607f7ed8615120a04e4729fbafa 2023-05-02 02:53:47,490 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2/8d86bdc24ade4df9963a5d3f077111d2 for region: 084b1607f7ed8615120a04e4729fbafa 2023-05-02 02:53:47,490 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2/9cf56291619249dda5dc07d66464b1af for region: 084b1607f7ed8615120a04e4729fbafa 2023-05-02 02:53:47,491 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2/a68c693682044c6c8576e930cd4eaf62 for region: 084b1607f7ed8615120a04e4729fbafa 2023-05-02 02:53:47,491 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2/b5f2f094012f478e9aa22a132932427f for region: 084b1607f7ed8615120a04e4729fbafa 2023-05-02 02:53:47,491 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2/cf84b8983146452aa1df04c6b4dfbbb6 for region: 084b1607f7ed8615120a04e4729fbafa 2023-05-02 02:53:47,492 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2/ea424b3182454ff5a1a8d6c3795602a2 for region: 084b1607f7ed8615120a04e4729fbafa 2023-05-02 02:53:47,492 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1/0a58882c16bc41b788b4deccd233becc for region: 084b1607f7ed8615120a04e4729fbafa 2023-05-02 02:53:47,492 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1/0cbb9fddcfd74f8c853f11e6ad27b8c1 for region: 084b1607f7ed8615120a04e4729fbafa 2023-05-02 02:53:47,492 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1/1893934900b2482ea071cead41006e9d for region: 084b1607f7ed8615120a04e4729fbafa 2023-05-02 02:53:47,493 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1/2cb812bd81a94565a0a1cf59a289bd05 for region: 084b1607f7ed8615120a04e4729fbafa 2023-05-02 02:53:47,493 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1/3663054878d64ef28b37a569207194d8 for region: 084b1607f7ed8615120a04e4729fbafa 2023-05-02 02:53:47,493 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1/7285a69c46ad4b738dfaa96fd16f3ad9 for region: 084b1607f7ed8615120a04e4729fbafa 2023-05-02 02:53:47,493 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1/6d14eb71c6f244cba148da66de9514b6 for region: 084b1607f7ed8615120a04e4729fbafa 2023-05-02 02:53:47,536 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2/a68c693682044c6c8576e930cd4eaf62, top=true 2023-05-02 02:53:47,538 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1/1893934900b2482ea071cead41006e9d, top=true 2023-05-02 02:53:47,560 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2/7fcf6b93d24a4fb5bb305342e41fd972, top=true 2023-05-02 02:53:47,561 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2/8d86bdc24ade4df9963a5d3f077111d2, top=true 2023-05-02 02:53:47,566 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1/0cbb9fddcfd74f8c853f11e6ad27b8c1, top=true 2023-05-02 02:53:47,575 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1/6d14eb71c6f244cba148da66de9514b6, top=true 2023-05-02 02:53:47,605 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/.splits/408e891cab9b919f4038b233ba34779d/cf1/testSplitTableRegionUnevenDaughter=084b1607f7ed8615120a04e4729fbafa-1893934900b2482ea071cead41006e9d for child: 408e891cab9b919f4038b233ba34779d, parent: 084b1607f7ed8615120a04e4729fbafa 2023-05-02 02:53:47,605 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1/1893934900b2482ea071cead41006e9d for region: 084b1607f7ed8615120a04e4729fbafa 2023-05-02 02:53:47,612 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/.splits/408e891cab9b919f4038b233ba34779d/cf2/testSplitTableRegionUnevenDaughter=084b1607f7ed8615120a04e4729fbafa-a68c693682044c6c8576e930cd4eaf62 for child: 408e891cab9b919f4038b233ba34779d, parent: 084b1607f7ed8615120a04e4729fbafa 2023-05-02 02:53:47,613 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2/a68c693682044c6c8576e930cd4eaf62 for region: 084b1607f7ed8615120a04e4729fbafa 2023-05-02 02:53:47,616 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/.splits/408e891cab9b919f4038b233ba34779d/cf1/testSplitTableRegionUnevenDaughter=084b1607f7ed8615120a04e4729fbafa-0cbb9fddcfd74f8c853f11e6ad27b8c1 for child: 408e891cab9b919f4038b233ba34779d, parent: 084b1607f7ed8615120a04e4729fbafa 2023-05-02 02:53:47,616 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1/0cbb9fddcfd74f8c853f11e6ad27b8c1 for region: 084b1607f7ed8615120a04e4729fbafa 2023-05-02 02:53:47,628 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/.splits/408e891cab9b919f4038b233ba34779d/cf1/testSplitTableRegionUnevenDaughter=084b1607f7ed8615120a04e4729fbafa-6d14eb71c6f244cba148da66de9514b6 for child: 408e891cab9b919f4038b233ba34779d, parent: 084b1607f7ed8615120a04e4729fbafa 2023-05-02 02:53:47,628 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1/6d14eb71c6f244cba148da66de9514b6 for region: 084b1607f7ed8615120a04e4729fbafa 2023-05-02 02:53:47,641 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/.splits/408e891cab9b919f4038b233ba34779d/cf2/testSplitTableRegionUnevenDaughter=084b1607f7ed8615120a04e4729fbafa-7fcf6b93d24a4fb5bb305342e41fd972 for child: 408e891cab9b919f4038b233ba34779d, parent: 084b1607f7ed8615120a04e4729fbafa 2023-05-02 02:53:47,641 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2/7fcf6b93d24a4fb5bb305342e41fd972 for region: 084b1607f7ed8615120a04e4729fbafa 2023-05-02 02:53:47,649 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/.splits/408e891cab9b919f4038b233ba34779d/cf2/testSplitTableRegionUnevenDaughter=084b1607f7ed8615120a04e4729fbafa-8d86bdc24ade4df9963a5d3f077111d2 for child: 408e891cab9b919f4038b233ba34779d, parent: 084b1607f7ed8615120a04e4729fbafa 2023-05-02 02:53:47,649 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2/8d86bdc24ade4df9963a5d3f077111d2 for region: 084b1607f7ed8615120a04e4729fbafa 2023-05-02 02:53:47,741 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2/b5f2f094012f478e9aa22a132932427f for region: 084b1607f7ed8615120a04e4729fbafa 2023-05-02 02:53:47,769 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2/cf84b8983146452aa1df04c6b4dfbbb6 for region: 084b1607f7ed8615120a04e4729fbafa 2023-05-02 02:53:47,785 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1/0a58882c16bc41b788b4deccd233becc for region: 084b1607f7ed8615120a04e4729fbafa 2023-05-02 02:53:47,792 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1/2cb812bd81a94565a0a1cf59a289bd05 for region: 084b1607f7ed8615120a04e4729fbafa 2023-05-02 02:53:47,800 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2/ea424b3182454ff5a1a8d6c3795602a2 for region: 084b1607f7ed8615120a04e4729fbafa 2023-05-02 02:53:48,032 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2/9cf56291619249dda5dc07d66464b1af for region: 084b1607f7ed8615120a04e4729fbafa 2023-05-02 02:53:48,202 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1/3663054878d64ef28b37a569207194d8 for region: 084b1607f7ed8615120a04e4729fbafa 2023-05-02 02:53:48,202 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1/7285a69c46ad4b738dfaa96fd16f3ad9 for region: 084b1607f7ed8615120a04e4729fbafa 2023-05-02 02:53:48,202 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=12 split storefiles for region 084b1607f7ed8615120a04e4729fbafa Daughter A: 8 storefiles, Daughter B: 14 storefiles. 2023-05-02 02:53:48,259 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/ea7683cd3436a62eb16418a3c875d563/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-05-02 02:53:48,261 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-05-02 02:53:48,264 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1682996025014.084b1607f7ed8615120a04e4729fbafa.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1682996028264"},{"qualifier":"splitA","vlen":70,"tag":[],"timestamp":"1682996028264"},{"qualifier":"splitB","vlen":70,"tag":[],"timestamp":"1682996028264"}]},"ts":"1682996028264"} 2023-05-02 02:53:48,264 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1682996027177.ea7683cd3436a62eb16418a3c875d563.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1682996028264"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682996028264"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682996028264"}]},"ts":"1682996028264"} 2023-05-02 02:53:48,264 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1682996027177.408e891cab9b919f4038b233ba34779d.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1682996028264"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682996028264"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682996028264"}]},"ts":"1682996028264"} 2023-05-02 02:53:48,313 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=ea7683cd3436a62eb16418a3c875d563, ASSIGN}, {pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=408e891cab9b919f4038b233ba34779d, ASSIGN}] 2023-05-02 02:53:48,314 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=408e891cab9b919f4038b233ba34779d, ASSIGN 2023-05-02 02:53:48,315 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=408e891cab9b919f4038b233ba34779d, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,38925,1682996021633; forceNewPlan=false, retain=false 2023-05-02 02:53:48,315 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=ea7683cd3436a62eb16418a3c875d563, ASSIGN 2023-05-02 02:53:48,316 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=ea7683cd3436a62eb16418a3c875d563, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,38925,1682996021633; forceNewPlan=false, retain=false 2023-05-02 02:53:48,465 INFO [jenkins-hbase4:41119] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-05-02 02:53:48,466 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=ea7683cd3436a62eb16418a3c875d563, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:48,466 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1682996027177.ea7683cd3436a62eb16418a3c875d563.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1682996028466"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1682996028466"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682996028466"}]},"ts":"1682996028466"} 2023-05-02 02:53:48,469 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=17, ppid=15, state=RUNNABLE; OpenRegionProcedure ea7683cd3436a62eb16418a3c875d563, server=jenkins-hbase4.apache.org,38925,1682996021633}] 2023-05-02 02:53:48,472 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=408e891cab9b919f4038b233ba34779d, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:48,473 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1682996027177.408e891cab9b919f4038b233ba34779d.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1682996028472"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1682996028472"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682996028472"}]},"ts":"1682996028472"} 2023-05-02 02:53:48,474 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=18, ppid=16, state=RUNNABLE; OpenRegionProcedure 408e891cab9b919f4038b233ba34779d, server=jenkins-hbase4.apache.org,38925,1682996021633}] 2023-05-02 02:53:48,627 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionUnevenDaughter,,1682996027177.ea7683cd3436a62eb16418a3c875d563. 2023-05-02 02:53:48,627 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => ea7683cd3436a62eb16418a3c875d563, NAME => 'testSplitTableRegionUnevenDaughter,,1682996027177.ea7683cd3436a62eb16418a3c875d563.', STARTKEY => '', ENDKEY => '26'} 2023-05-02 02:53:48,628 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-02 02:53:48,628 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter ea7683cd3436a62eb16418a3c875d563 2023-05-02 02:53:48,628 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionUnevenDaughter,,1682996027177.ea7683cd3436a62eb16418a3c875d563.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-02 02:53:48,628 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for ea7683cd3436a62eb16418a3c875d563 2023-05-02 02:53:48,628 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for ea7683cd3436a62eb16418a3c875d563 2023-05-02 02:53:48,635 INFO [StoreOpener-ea7683cd3436a62eb16418a3c875d563-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region ea7683cd3436a62eb16418a3c875d563 2023-05-02 02:53:48,637 DEBUG [StoreOpener-ea7683cd3436a62eb16418a3c875d563-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/ea7683cd3436a62eb16418a3c875d563/cf1 2023-05-02 02:53:48,637 DEBUG [StoreOpener-ea7683cd3436a62eb16418a3c875d563-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/ea7683cd3436a62eb16418a3c875d563/cf1 2023-05-02 02:53:48,638 INFO [StoreOpener-ea7683cd3436a62eb16418a3c875d563-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ea7683cd3436a62eb16418a3c875d563 columnFamilyName cf1 2023-05-02 02:53:48,683 DEBUG [StoreOpener-ea7683cd3436a62eb16418a3c875d563-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/ea7683cd3436a62eb16418a3c875d563/cf1/0a58882c16bc41b788b4deccd233becc.084b1607f7ed8615120a04e4729fbafa->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1/0a58882c16bc41b788b4deccd233becc-bottom 2023-05-02 02:53:48,694 DEBUG [StoreOpener-ea7683cd3436a62eb16418a3c875d563-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/ea7683cd3436a62eb16418a3c875d563/cf1/2cb812bd81a94565a0a1cf59a289bd05.084b1607f7ed8615120a04e4729fbafa->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1/2cb812bd81a94565a0a1cf59a289bd05-bottom 2023-05-02 02:53:48,703 DEBUG [StoreOpener-ea7683cd3436a62eb16418a3c875d563-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/ea7683cd3436a62eb16418a3c875d563/cf1/3663054878d64ef28b37a569207194d8.084b1607f7ed8615120a04e4729fbafa->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1/3663054878d64ef28b37a569207194d8-bottom 2023-05-02 02:53:48,711 DEBUG [StoreOpener-ea7683cd3436a62eb16418a3c875d563-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/ea7683cd3436a62eb16418a3c875d563/cf1/7285a69c46ad4b738dfaa96fd16f3ad9.084b1607f7ed8615120a04e4729fbafa->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1/7285a69c46ad4b738dfaa96fd16f3ad9-bottom 2023-05-02 02:53:48,712 INFO [StoreOpener-ea7683cd3436a62eb16418a3c875d563-1] regionserver.HStore(310): Store=ea7683cd3436a62eb16418a3c875d563/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-02 02:53:48,712 INFO [StoreOpener-ea7683cd3436a62eb16418a3c875d563-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region ea7683cd3436a62eb16418a3c875d563 2023-05-02 02:53:48,714 DEBUG [StoreOpener-ea7683cd3436a62eb16418a3c875d563-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/ea7683cd3436a62eb16418a3c875d563/cf2 2023-05-02 02:53:48,715 DEBUG [StoreOpener-ea7683cd3436a62eb16418a3c875d563-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/ea7683cd3436a62eb16418a3c875d563/cf2 2023-05-02 02:53:48,715 INFO [StoreOpener-ea7683cd3436a62eb16418a3c875d563-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ea7683cd3436a62eb16418a3c875d563 columnFamilyName cf2 2023-05-02 02:53:48,733 DEBUG [StoreOpener-ea7683cd3436a62eb16418a3c875d563-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/ea7683cd3436a62eb16418a3c875d563/cf2/9cf56291619249dda5dc07d66464b1af.084b1607f7ed8615120a04e4729fbafa->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2/9cf56291619249dda5dc07d66464b1af-bottom 2023-05-02 02:53:48,742 DEBUG [StoreOpener-ea7683cd3436a62eb16418a3c875d563-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/ea7683cd3436a62eb16418a3c875d563/cf2/b5f2f094012f478e9aa22a132932427f.084b1607f7ed8615120a04e4729fbafa->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2/b5f2f094012f478e9aa22a132932427f-bottom 2023-05-02 02:53:48,749 DEBUG [StoreOpener-ea7683cd3436a62eb16418a3c875d563-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/ea7683cd3436a62eb16418a3c875d563/cf2/cf84b8983146452aa1df04c6b4dfbbb6.084b1607f7ed8615120a04e4729fbafa->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2/cf84b8983146452aa1df04c6b4dfbbb6-bottom 2023-05-02 02:53:48,756 DEBUG [StoreOpener-ea7683cd3436a62eb16418a3c875d563-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/ea7683cd3436a62eb16418a3c875d563/cf2/ea424b3182454ff5a1a8d6c3795602a2.084b1607f7ed8615120a04e4729fbafa->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2/ea424b3182454ff5a1a8d6c3795602a2-bottom 2023-05-02 02:53:48,756 INFO [StoreOpener-ea7683cd3436a62eb16418a3c875d563-1] regionserver.HStore(310): Store=ea7683cd3436a62eb16418a3c875d563/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-02 02:53:48,757 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/ea7683cd3436a62eb16418a3c875d563 2023-05-02 02:53:48,759 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/ea7683cd3436a62eb16418a3c875d563 2023-05-02 02:53:48,763 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-02 02:53:48,765 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for ea7683cd3436a62eb16418a3c875d563 2023-05-02 02:53:48,766 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened ea7683cd3436a62eb16418a3c875d563; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60333844, jitterRate=-0.10095566511154175}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-02 02:53:48,766 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for ea7683cd3436a62eb16418a3c875d563: 2023-05-02 02:53:48,767 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1682996027177.ea7683cd3436a62eb16418a3c875d563., pid=17, masterSystemTime=1682996028621 2023-05-02 02:53:48,770 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-02 02:53:48,770 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-05-02 02:53:48,771 DEBUG [RS:1;jenkins-hbase4:38925-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-05-02 02:53:48,771 DEBUG [RS:1;jenkins-hbase4:38925-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-05-02 02:53:48,773 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1682996027177.ea7683cd3436a62eb16418a3c875d563. 2023-05-02 02:53:48,773 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionUnevenDaughter,,1682996027177.ea7683cd3436a62eb16418a3c875d563. 2023-05-02 02:53:48,773 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionUnevenDaughter,26,1682996027177.408e891cab9b919f4038b233ba34779d. 2023-05-02 02:53:48,773 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 408e891cab9b919f4038b233ba34779d, NAME => 'testSplitTableRegionUnevenDaughter,26,1682996027177.408e891cab9b919f4038b233ba34779d.', STARTKEY => '26', ENDKEY => ''} 2023-05-02 02:53:48,773 INFO [RS:1;jenkins-hbase4:38925-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1682996027177.ea7683cd3436a62eb16418a3c875d563. 2023-05-02 02:53:48,773 INFO [RS:1;jenkins-hbase4:38925-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1682996027177.ea7683cd3436a62eb16418a3c875d563. 2023-05-02 02:53:48,773 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-02 02:53:48,773 DEBUG [RS:1;jenkins-hbase4:38925-longCompactions-0] regionserver.HStore(1912): ea7683cd3436a62eb16418a3c875d563/cf2 is initiating minor compaction (all files) 2023-05-02 02:53:48,774 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=ea7683cd3436a62eb16418a3c875d563, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:48,774 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter 408e891cab9b919f4038b233ba34779d 2023-05-02 02:53:48,774 INFO [RS:1;jenkins-hbase4:38925-longCompactions-0] regionserver.HRegion(2259): Starting compaction of ea7683cd3436a62eb16418a3c875d563/cf2 in testSplitTableRegionUnevenDaughter,,1682996027177.ea7683cd3436a62eb16418a3c875d563. 2023-05-02 02:53:48,774 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionUnevenDaughter,26,1682996027177.408e891cab9b919f4038b233ba34779d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-02 02:53:48,773 DEBUG [RS:1;jenkins-hbase4:38925-shortCompactions-0] regionserver.HStore(1912): ea7683cd3436a62eb16418a3c875d563/cf1 is initiating minor compaction (all files) 2023-05-02 02:53:48,774 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1682996027177.ea7683cd3436a62eb16418a3c875d563.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1682996028773"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1682996028773"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1682996028773"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682996028773"}]},"ts":"1682996028773"} 2023-05-02 02:53:48,774 INFO [RS:1;jenkins-hbase4:38925-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of ea7683cd3436a62eb16418a3c875d563/cf1 in testSplitTableRegionUnevenDaughter,,1682996027177.ea7683cd3436a62eb16418a3c875d563. 2023-05-02 02:53:48,774 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 408e891cab9b919f4038b233ba34779d 2023-05-02 02:53:48,775 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 408e891cab9b919f4038b233ba34779d 2023-05-02 02:53:48,775 INFO [RS:1;jenkins-hbase4:38925-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/ea7683cd3436a62eb16418a3c875d563/cf2/b5f2f094012f478e9aa22a132932427f.084b1607f7ed8615120a04e4729fbafa->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2/b5f2f094012f478e9aa22a132932427f-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/ea7683cd3436a62eb16418a3c875d563/cf2/ea424b3182454ff5a1a8d6c3795602a2.084b1607f7ed8615120a04e4729fbafa->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2/ea424b3182454ff5a1a8d6c3795602a2-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/ea7683cd3436a62eb16418a3c875d563/cf2/9cf56291619249dda5dc07d66464b1af.084b1607f7ed8615120a04e4729fbafa->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2/9cf56291619249dda5dc07d66464b1af-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/ea7683cd3436a62eb16418a3c875d563/cf2/cf84b8983146452aa1df04c6b4dfbbb6.084b1607f7ed8615120a04e4729fbafa->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2/cf84b8983146452aa1df04c6b4dfbbb6-bottom] into tmpdir=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/ea7683cd3436a62eb16418a3c875d563/.tmp, totalSize=19.9 K 2023-05-02 02:53:48,775 INFO [RS:1;jenkins-hbase4:38925-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/ea7683cd3436a62eb16418a3c875d563/cf1/3663054878d64ef28b37a569207194d8.084b1607f7ed8615120a04e4729fbafa->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1/3663054878d64ef28b37a569207194d8-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/ea7683cd3436a62eb16418a3c875d563/cf1/2cb812bd81a94565a0a1cf59a289bd05.084b1607f7ed8615120a04e4729fbafa->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1/2cb812bd81a94565a0a1cf59a289bd05-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/ea7683cd3436a62eb16418a3c875d563/cf1/7285a69c46ad4b738dfaa96fd16f3ad9.084b1607f7ed8615120a04e4729fbafa->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1/7285a69c46ad4b738dfaa96fd16f3ad9-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/ea7683cd3436a62eb16418a3c875d563/cf1/0a58882c16bc41b788b4deccd233becc.084b1607f7ed8615120a04e4729fbafa->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1/0a58882c16bc41b788b4deccd233becc-bottom] into tmpdir=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/ea7683cd3436a62eb16418a3c875d563/.tmp, totalSize=19.9 K 2023-05-02 02:53:48,777 DEBUG [RS:1;jenkins-hbase4:38925-longCompactions-0] compactions.Compactor(207): Compacting b5f2f094012f478e9aa22a132932427f.084b1607f7ed8615120a04e4729fbafa, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1682996025522 2023-05-02 02:53:48,778 DEBUG [RS:1;jenkins-hbase4:38925-shortCompactions-0] compactions.Compactor(207): Compacting 3663054878d64ef28b37a569207194d8.084b1607f7ed8615120a04e4729fbafa, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1682996025522 2023-05-02 02:53:48,777 INFO [StoreOpener-408e891cab9b919f4038b233ba34779d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 408e891cab9b919f4038b233ba34779d 2023-05-02 02:53:48,779 DEBUG [RS:1;jenkins-hbase4:38925-longCompactions-0] compactions.Compactor(207): Compacting ea424b3182454ff5a1a8d6c3795602a2.084b1607f7ed8615120a04e4729fbafa, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1682996025891 2023-05-02 02:53:48,780 DEBUG [RS:1;jenkins-hbase4:38925-shortCompactions-0] compactions.Compactor(207): Compacting 2cb812bd81a94565a0a1cf59a289bd05.084b1607f7ed8615120a04e4729fbafa, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1682996025891 2023-05-02 02:53:48,780 DEBUG [StoreOpener-408e891cab9b919f4038b233ba34779d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf1 2023-05-02 02:53:48,780 DEBUG [RS:1;jenkins-hbase4:38925-shortCompactions-0] compactions.Compactor(207): Compacting 7285a69c46ad4b738dfaa96fd16f3ad9.084b1607f7ed8615120a04e4729fbafa, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1682996026155 2023-05-02 02:53:48,780 DEBUG [StoreOpener-408e891cab9b919f4038b233ba34779d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf1 2023-05-02 02:53:48,780 DEBUG [RS:1;jenkins-hbase4:38925-longCompactions-0] compactions.Compactor(207): Compacting 9cf56291619249dda5dc07d66464b1af.084b1607f7ed8615120a04e4729fbafa, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1682996026155 2023-05-02 02:53:48,781 INFO [StoreOpener-408e891cab9b919f4038b233ba34779d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 408e891cab9b919f4038b233ba34779d columnFamilyName cf1 2023-05-02 02:53:48,781 DEBUG [RS:1;jenkins-hbase4:38925-shortCompactions-0] compactions.Compactor(207): Compacting 0a58882c16bc41b788b4deccd233becc.084b1607f7ed8615120a04e4729fbafa, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1682996026408 2023-05-02 02:53:48,782 DEBUG [RS:1;jenkins-hbase4:38925-longCompactions-0] compactions.Compactor(207): Compacting cf84b8983146452aa1df04c6b4dfbbb6.084b1607f7ed8615120a04e4729fbafa, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1682996026408 2023-05-02 02:53:48,783 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=15 2023-05-02 02:53:48,783 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=15, state=SUCCESS; OpenRegionProcedure ea7683cd3436a62eb16418a3c875d563, server=jenkins-hbase4.apache.org,38925,1682996021633 in 309 msec 2023-05-02 02:53:48,786 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=15, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=ea7683cd3436a62eb16418a3c875d563, ASSIGN in 472 msec 2023-05-02 02:53:48,806 DEBUG [StoreOpener-408e891cab9b919f4038b233ba34779d-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf1/0a58882c16bc41b788b4deccd233becc.084b1607f7ed8615120a04e4729fbafa->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1/0a58882c16bc41b788b4deccd233becc-top 2023-05-02 02:53:48,825 DEBUG [StoreOpener-408e891cab9b919f4038b233ba34779d-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf1/2cb812bd81a94565a0a1cf59a289bd05.084b1607f7ed8615120a04e4729fbafa->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1/2cb812bd81a94565a0a1cf59a289bd05-top 2023-05-02 02:53:48,826 INFO [RS:1;jenkins-hbase4:38925-longCompactions-0] throttle.PressureAwareThroughputController(145): ea7683cd3436a62eb16418a3c875d563#cf2#compaction#14 average throughput is 0.21 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-02 02:53:48,826 INFO [RS:1;jenkins-hbase4:38925-shortCompactions-0] throttle.PressureAwareThroughputController(145): ea7683cd3436a62eb16418a3c875d563#cf1#compaction#15 average throughput is 0.21 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-02 02:53:48,834 DEBUG [StoreOpener-408e891cab9b919f4038b233ba34779d-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf1/3663054878d64ef28b37a569207194d8.084b1607f7ed8615120a04e4729fbafa->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1/3663054878d64ef28b37a569207194d8-top 2023-05-02 02:53:48,840 DEBUG [StoreOpener-408e891cab9b919f4038b233ba34779d-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf1/7285a69c46ad4b738dfaa96fd16f3ad9.084b1607f7ed8615120a04e4729fbafa->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1/7285a69c46ad4b738dfaa96fd16f3ad9-top 2023-05-02 02:53:48,848 DEBUG [StoreOpener-408e891cab9b919f4038b233ba34779d-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf1/testSplitTableRegionUnevenDaughter=084b1607f7ed8615120a04e4729fbafa-0cbb9fddcfd74f8c853f11e6ad27b8c1 2023-05-02 02:53:48,859 DEBUG [StoreOpener-408e891cab9b919f4038b233ba34779d-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf1/testSplitTableRegionUnevenDaughter=084b1607f7ed8615120a04e4729fbafa-1893934900b2482ea071cead41006e9d 2023-05-02 02:53:48,864 DEBUG [RS:1;jenkins-hbase4:38925-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/ea7683cd3436a62eb16418a3c875d563/.tmp/cf1/fd2597bbde7e4768b1f8e7bae14ba9af as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/ea7683cd3436a62eb16418a3c875d563/cf1/fd2597bbde7e4768b1f8e7bae14ba9af 2023-05-02 02:53:48,868 DEBUG [RS:1;jenkins-hbase4:38925-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/ea7683cd3436a62eb16418a3c875d563/.tmp/cf2/fd29bac7a8e14d8190522b2883a65b5f as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/ea7683cd3436a62eb16418a3c875d563/cf2/fd29bac7a8e14d8190522b2883a65b5f 2023-05-02 02:53:48,874 DEBUG [StoreOpener-408e891cab9b919f4038b233ba34779d-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf1/testSplitTableRegionUnevenDaughter=084b1607f7ed8615120a04e4729fbafa-6d14eb71c6f244cba148da66de9514b6 2023-05-02 02:53:48,875 INFO [StoreOpener-408e891cab9b919f4038b233ba34779d-1] regionserver.HStore(310): Store=408e891cab9b919f4038b233ba34779d/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-02 02:53:48,876 INFO [StoreOpener-408e891cab9b919f4038b233ba34779d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 408e891cab9b919f4038b233ba34779d 2023-05-02 02:53:48,877 DEBUG [StoreOpener-408e891cab9b919f4038b233ba34779d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf2 2023-05-02 02:53:48,877 DEBUG [StoreOpener-408e891cab9b919f4038b233ba34779d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf2 2023-05-02 02:53:48,878 INFO [StoreOpener-408e891cab9b919f4038b233ba34779d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 408e891cab9b919f4038b233ba34779d columnFamilyName cf2 2023-05-02 02:53:48,889 INFO [RS:1;jenkins-hbase4:38925-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in ea7683cd3436a62eb16418a3c875d563/cf1 of ea7683cd3436a62eb16418a3c875d563 into fd2597bbde7e4768b1f8e7bae14ba9af(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-02 02:53:48,889 INFO [RS:1;jenkins-hbase4:38925-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in ea7683cd3436a62eb16418a3c875d563/cf2 of ea7683cd3436a62eb16418a3c875d563 into fd29bac7a8e14d8190522b2883a65b5f(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-02 02:53:48,891 DEBUG [RS:1;jenkins-hbase4:38925-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for ea7683cd3436a62eb16418a3c875d563: 2023-05-02 02:53:48,891 DEBUG [RS:1;jenkins-hbase4:38925-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for ea7683cd3436a62eb16418a3c875d563: 2023-05-02 02:53:48,891 INFO [RS:1;jenkins-hbase4:38925-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1682996027177.ea7683cd3436a62eb16418a3c875d563., storeName=ea7683cd3436a62eb16418a3c875d563/cf2, priority=12, startTime=1682996028770; duration=0sec 2023-05-02 02:53:48,891 INFO [RS:1;jenkins-hbase4:38925-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1682996027177.ea7683cd3436a62eb16418a3c875d563., storeName=ea7683cd3436a62eb16418a3c875d563/cf1, priority=12, startTime=1682996028769; duration=0sec 2023-05-02 02:53:48,892 DEBUG [RS:1;jenkins-hbase4:38925-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-02 02:53:48,892 DEBUG [RS:1;jenkins-hbase4:38925-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-02 02:53:48,896 DEBUG [StoreOpener-408e891cab9b919f4038b233ba34779d-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf2/9cf56291619249dda5dc07d66464b1af.084b1607f7ed8615120a04e4729fbafa->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2/9cf56291619249dda5dc07d66464b1af-top 2023-05-02 02:53:48,901 DEBUG [StoreOpener-408e891cab9b919f4038b233ba34779d-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf2/b5f2f094012f478e9aa22a132932427f.084b1607f7ed8615120a04e4729fbafa->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2/b5f2f094012f478e9aa22a132932427f-top 2023-05-02 02:53:48,908 DEBUG [StoreOpener-408e891cab9b919f4038b233ba34779d-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf2/cf84b8983146452aa1df04c6b4dfbbb6.084b1607f7ed8615120a04e4729fbafa->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2/cf84b8983146452aa1df04c6b4dfbbb6-top 2023-05-02 02:53:48,913 DEBUG [StoreOpener-408e891cab9b919f4038b233ba34779d-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf2/ea424b3182454ff5a1a8d6c3795602a2.084b1607f7ed8615120a04e4729fbafa->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2/ea424b3182454ff5a1a8d6c3795602a2-top 2023-05-02 02:53:48,918 DEBUG [StoreOpener-408e891cab9b919f4038b233ba34779d-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf2/testSplitTableRegionUnevenDaughter=084b1607f7ed8615120a04e4729fbafa-7fcf6b93d24a4fb5bb305342e41fd972 2023-05-02 02:53:48,923 DEBUG [StoreOpener-408e891cab9b919f4038b233ba34779d-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf2/testSplitTableRegionUnevenDaughter=084b1607f7ed8615120a04e4729fbafa-8d86bdc24ade4df9963a5d3f077111d2 2023-05-02 02:53:48,926 DEBUG [StoreOpener-408e891cab9b919f4038b233ba34779d-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf2/testSplitTableRegionUnevenDaughter=084b1607f7ed8615120a04e4729fbafa-a68c693682044c6c8576e930cd4eaf62 2023-05-02 02:53:48,927 INFO [StoreOpener-408e891cab9b919f4038b233ba34779d-1] regionserver.HStore(310): Store=408e891cab9b919f4038b233ba34779d/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-02 02:53:48,927 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d 2023-05-02 02:53:48,930 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d 2023-05-02 02:53:48,931 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-02 02:53:48,933 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 408e891cab9b919f4038b233ba34779d 2023-05-02 02:53:48,934 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 408e891cab9b919f4038b233ba34779d; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63405074, jitterRate=-0.05519077181816101}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-02 02:53:48,934 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 408e891cab9b919f4038b233ba34779d: 2023-05-02 02:53:48,935 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,26,1682996027177.408e891cab9b919f4038b233ba34779d., pid=18, masterSystemTime=1682996028621 2023-05-02 02:53:48,935 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-02 02:53:48,936 DEBUG [RS:1;jenkins-hbase4:38925-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-02 02:53:48,936 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-05-02 02:53:48,936 DEBUG [RS:1;jenkins-hbase4:38925-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-02 02:53:48,936 INFO [RS:1;jenkins-hbase4:38925-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1682996027177.408e891cab9b919f4038b233ba34779d. 2023-05-02 02:53:48,936 DEBUG [RS:1;jenkins-hbase4:38925-shortCompactions-0] regionserver.HStore(1912): 408e891cab9b919f4038b233ba34779d/cf1 is initiating minor compaction (all files) 2023-05-02 02:53:48,936 INFO [RS:1;jenkins-hbase4:38925-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1682996027177.408e891cab9b919f4038b233ba34779d. 2023-05-02 02:53:48,936 INFO [RS:1;jenkins-hbase4:38925-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 408e891cab9b919f4038b233ba34779d/cf1 in testSplitTableRegionUnevenDaughter,26,1682996027177.408e891cab9b919f4038b233ba34779d. 2023-05-02 02:53:48,936 DEBUG [RS:1;jenkins-hbase4:38925-longCompactions-0] regionserver.HStore(1912): 408e891cab9b919f4038b233ba34779d/cf2 is initiating minor compaction (all files) 2023-05-02 02:53:48,937 INFO [RS:1;jenkins-hbase4:38925-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 408e891cab9b919f4038b233ba34779d/cf2 in testSplitTableRegionUnevenDaughter,26,1682996027177.408e891cab9b919f4038b233ba34779d. 2023-05-02 02:53:48,937 INFO [RS:1;jenkins-hbase4:38925-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf1/3663054878d64ef28b37a569207194d8.084b1607f7ed8615120a04e4729fbafa->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1/3663054878d64ef28b37a569207194d8-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf1/2cb812bd81a94565a0a1cf59a289bd05.084b1607f7ed8615120a04e4729fbafa->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1/2cb812bd81a94565a0a1cf59a289bd05-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf1/7285a69c46ad4b738dfaa96fd16f3ad9.084b1607f7ed8615120a04e4729fbafa->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1/7285a69c46ad4b738dfaa96fd16f3ad9-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf1/0a58882c16bc41b788b4deccd233becc.084b1607f7ed8615120a04e4729fbafa->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1/0a58882c16bc41b788b4deccd233becc-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf1/testSplitTableRegionUnevenDaughter=084b1607f7ed8615120a04e4729fbafa-1893934900b2482ea071cead41006e9d, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf1/testSplitTableRegionUnevenDaughter=084b1607f7ed8615120a04e4729fbafa-0cbb9fddcfd74f8c853f11e6ad27b8c1, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf1/testSplitTableRegionUnevenDaughter=084b1607f7ed8615120a04e4729fbafa-6d14eb71c6f244cba148da66de9514b6] into tmpdir=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/.tmp, totalSize=34.9 K 2023-05-02 02:53:48,937 INFO [RS:1;jenkins-hbase4:38925-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf2/b5f2f094012f478e9aa22a132932427f.084b1607f7ed8615120a04e4729fbafa->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2/b5f2f094012f478e9aa22a132932427f-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf2/ea424b3182454ff5a1a8d6c3795602a2.084b1607f7ed8615120a04e4729fbafa->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2/ea424b3182454ff5a1a8d6c3795602a2-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf2/9cf56291619249dda5dc07d66464b1af.084b1607f7ed8615120a04e4729fbafa->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2/9cf56291619249dda5dc07d66464b1af-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf2/cf84b8983146452aa1df04c6b4dfbbb6.084b1607f7ed8615120a04e4729fbafa->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2/cf84b8983146452aa1df04c6b4dfbbb6-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf2/testSplitTableRegionUnevenDaughter=084b1607f7ed8615120a04e4729fbafa-8d86bdc24ade4df9963a5d3f077111d2, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf2/testSplitTableRegionUnevenDaughter=084b1607f7ed8615120a04e4729fbafa-7fcf6b93d24a4fb5bb305342e41fd972, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf2/testSplitTableRegionUnevenDaughter=084b1607f7ed8615120a04e4729fbafa-a68c693682044c6c8576e930cd4eaf62] into tmpdir=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/.tmp, totalSize=34.9 K 2023-05-02 02:53:48,937 DEBUG [RS:1;jenkins-hbase4:38925-shortCompactions-0] compactions.Compactor(207): Compacting 3663054878d64ef28b37a569207194d8.084b1607f7ed8615120a04e4729fbafa, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1682996025522 2023-05-02 02:53:48,939 DEBUG [RS:1;jenkins-hbase4:38925-longCompactions-0] compactions.Compactor(207): Compacting b5f2f094012f478e9aa22a132932427f.084b1607f7ed8615120a04e4729fbafa, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1682996025522 2023-05-02 02:53:48,940 DEBUG [RS:1;jenkins-hbase4:38925-shortCompactions-0] compactions.Compactor(207): Compacting 2cb812bd81a94565a0a1cf59a289bd05.084b1607f7ed8615120a04e4729fbafa, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1682996025891 2023-05-02 02:53:48,940 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,26,1682996027177.408e891cab9b919f4038b233ba34779d. 2023-05-02 02:53:48,940 DEBUG [RS:1;jenkins-hbase4:38925-longCompactions-0] compactions.Compactor(207): Compacting ea424b3182454ff5a1a8d6c3795602a2.084b1607f7ed8615120a04e4729fbafa, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1682996025891 2023-05-02 02:53:48,940 DEBUG [RS:1;jenkins-hbase4:38925-shortCompactions-0] compactions.Compactor(207): Compacting 7285a69c46ad4b738dfaa96fd16f3ad9.084b1607f7ed8615120a04e4729fbafa, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1682996026155 2023-05-02 02:53:48,940 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionUnevenDaughter,26,1682996027177.408e891cab9b919f4038b233ba34779d. 2023-05-02 02:53:48,941 DEBUG [RS:1;jenkins-hbase4:38925-longCompactions-0] compactions.Compactor(207): Compacting 9cf56291619249dda5dc07d66464b1af.084b1607f7ed8615120a04e4729fbafa, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1682996026155 2023-05-02 02:53:48,941 DEBUG [RS:1;jenkins-hbase4:38925-shortCompactions-0] compactions.Compactor(207): Compacting 0a58882c16bc41b788b4deccd233becc.084b1607f7ed8615120a04e4729fbafa, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1682996026408 2023-05-02 02:53:48,941 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=408e891cab9b919f4038b233ba34779d, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:48,942 DEBUG [RS:1;jenkins-hbase4:38925-longCompactions-0] compactions.Compactor(207): Compacting cf84b8983146452aa1df04c6b4dfbbb6.084b1607f7ed8615120a04e4729fbafa, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1682996026408 2023-05-02 02:53:48,942 DEBUG [RS:1;jenkins-hbase4:38925-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=084b1607f7ed8615120a04e4729fbafa-1893934900b2482ea071cead41006e9d, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1682996026665 2023-05-02 02:53:48,942 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,26,1682996027177.408e891cab9b919f4038b233ba34779d.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1682996028941"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1682996028941"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1682996028941"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682996028941"}]},"ts":"1682996028941"} 2023-05-02 02:53:48,942 DEBUG [RS:1;jenkins-hbase4:38925-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=084b1607f7ed8615120a04e4729fbafa-8d86bdc24ade4df9963a5d3f077111d2, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1682996026665 2023-05-02 02:53:48,942 DEBUG [RS:1;jenkins-hbase4:38925-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=084b1607f7ed8615120a04e4729fbafa-0cbb9fddcfd74f8c853f11e6ad27b8c1, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1682996026920 2023-05-02 02:53:48,944 DEBUG [RS:1;jenkins-hbase4:38925-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=084b1607f7ed8615120a04e4729fbafa-7fcf6b93d24a4fb5bb305342e41fd972, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1682996026920 2023-05-02 02:53:48,944 DEBUG [RS:1;jenkins-hbase4:38925-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=084b1607f7ed8615120a04e4729fbafa-6d14eb71c6f244cba148da66de9514b6, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1682996027155 2023-05-02 02:53:48,945 DEBUG [RS:1;jenkins-hbase4:38925-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=084b1607f7ed8615120a04e4729fbafa-a68c693682044c6c8576e930cd4eaf62, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1682996027155 2023-05-02 02:53:48,949 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=18, resume processing ppid=16 2023-05-02 02:53:48,949 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=18, ppid=16, state=SUCCESS; OpenRegionProcedure 408e891cab9b919f4038b233ba34779d, server=jenkins-hbase4.apache.org,38925,1682996021633 in 471 msec 2023-05-02 02:53:48,952 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=16, resume processing ppid=12 2023-05-02 02:53:48,952 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=408e891cab9b919f4038b233ba34779d, ASSIGN in 637 msec 2023-05-02 02:53:48,956 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=084b1607f7ed8615120a04e4729fbafa, daughterA=ea7683cd3436a62eb16418a3c875d563, daughterB=408e891cab9b919f4038b233ba34779d in 1.7760 sec 2023-05-02 02:53:48,961 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-05-02 02:53:48,979 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38925] ipc.CallRunner(144): callId: 106 service: ClientService methodName: Scan size: 157 connection: 172.31.14.131:57932 deadline: 1682996088978, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionUnevenDaughter,,1682996025014.084b1607f7ed8615120a04e4729fbafa. is not online on jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:48,982 INFO [RS:1;jenkins-hbase4:38925-longCompactions-0] throttle.PressureAwareThroughputController(145): 408e891cab9b919f4038b233ba34779d#cf2#compaction#16 average throughput is 0.64 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-02 02:53:48,997 INFO [RS:1;jenkins-hbase4:38925-shortCompactions-0] throttle.PressureAwareThroughputController(145): 408e891cab9b919f4038b233ba34779d#cf1#compaction#17 average throughput is 0.32 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-02 02:53:49,016 DEBUG [RS:1;jenkins-hbase4:38925-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/.tmp/cf2/cdd2f69c524441fe935977abb83c9c50 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf2/cdd2f69c524441fe935977abb83c9c50 2023-05-02 02:53:49,024 INFO [RS:1;jenkins-hbase4:38925-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 408e891cab9b919f4038b233ba34779d/cf2 of 408e891cab9b919f4038b233ba34779d into cdd2f69c524441fe935977abb83c9c50(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-02 02:53:49,024 DEBUG [RS:1;jenkins-hbase4:38925-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 408e891cab9b919f4038b233ba34779d: 2023-05-02 02:53:49,024 INFO [RS:1;jenkins-hbase4:38925-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1682996027177.408e891cab9b919f4038b233ba34779d., storeName=408e891cab9b919f4038b233ba34779d/cf2, priority=9, startTime=1682996028936; duration=0sec 2023-05-02 02:53:49,024 DEBUG [RS:1;jenkins-hbase4:38925-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-02 02:53:49,037 DEBUG [RS:1;jenkins-hbase4:38925-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/.tmp/cf1/0da696f482c841fc85dde2d9dde1afed as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf1/0da696f482c841fc85dde2d9dde1afed 2023-05-02 02:53:49,048 INFO [RS:1;jenkins-hbase4:38925-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 408e891cab9b919f4038b233ba34779d/cf1 of 408e891cab9b919f4038b233ba34779d into 0da696f482c841fc85dde2d9dde1afed(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-02 02:53:49,048 DEBUG [RS:1;jenkins-hbase4:38925-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 408e891cab9b919f4038b233ba34779d: 2023-05-02 02:53:49,048 INFO [RS:1;jenkins-hbase4:38925-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1682996027177.408e891cab9b919f4038b233ba34779d., storeName=408e891cab9b919f4038b233ba34779d/cf1, priority=9, startTime=1682996028935; duration=0sec 2023-05-02 02:53:49,049 DEBUG [RS:1;jenkins-hbase4:38925-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-02 02:53:49,055 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-05-02 02:53:49,058 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-05-02 02:53:49,126 DEBUG [Listener at localhost/37119] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-05-02 02:53:49,136 INFO [Listener at localhost/37119] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-05-02 02:53:49,139 DEBUG [Listener at localhost/37119] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-05-02 02:53:49,167 WARN [Listener at localhost/37119] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-02 02:53:49,179 INFO [Listener at localhost/37119] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionUnevenDaughter 2023-05-02 02:53:49,185 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testSplitTableRegionUnevenDaughter 2023-05-02 02:53:49,195 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:53:49,201 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682996029201"}]},"ts":"1682996029201"} 2023-05-02 02:53:49,203 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLING in hbase:meta 2023-05-02 02:53:49,208 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-05-02 02:53:49,209 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionUnevenDaughter to state=DISABLING 2023-05-02 02:53:49,212 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=ea7683cd3436a62eb16418a3c875d563, UNASSIGN}, {pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=408e891cab9b919f4038b233ba34779d, UNASSIGN}] 2023-05-02 02:53:49,214 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=408e891cab9b919f4038b233ba34779d, UNASSIGN 2023-05-02 02:53:49,215 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=408e891cab9b919f4038b233ba34779d, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:49,215 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1682996027177.408e891cab9b919f4038b233ba34779d.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1682996029215"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1682996029215"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682996029215"}]},"ts":"1682996029215"} 2023-05-02 02:53:49,217 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=22, ppid=21, state=RUNNABLE; CloseRegionProcedure 408e891cab9b919f4038b233ba34779d, server=jenkins-hbase4.apache.org,38925,1682996021633}] 2023-05-02 02:53:49,221 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=ea7683cd3436a62eb16418a3c875d563, UNASSIGN 2023-05-02 02:53:49,222 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=ea7683cd3436a62eb16418a3c875d563, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:49,222 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1682996027177.ea7683cd3436a62eb16418a3c875d563.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1682996029222"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1682996029222"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682996029222"}]},"ts":"1682996029222"} 2023-05-02 02:53:49,224 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=23, ppid=20, state=RUNNABLE; CloseRegionProcedure ea7683cd3436a62eb16418a3c875d563, server=jenkins-hbase4.apache.org,38925,1682996021633}] 2023-05-02 02:53:49,313 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-05-02 02:53:49,370 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 408e891cab9b919f4038b233ba34779d 2023-05-02 02:53:49,370 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 408e891cab9b919f4038b233ba34779d, disabling compactions & flushes 2023-05-02 02:53:49,370 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionUnevenDaughter,26,1682996027177.408e891cab9b919f4038b233ba34779d. 2023-05-02 02:53:49,370 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,26,1682996027177.408e891cab9b919f4038b233ba34779d. 2023-05-02 02:53:49,370 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionUnevenDaughter,26,1682996027177.408e891cab9b919f4038b233ba34779d. after waiting 0 ms 2023-05-02 02:53:49,370 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionUnevenDaughter,26,1682996027177.408e891cab9b919f4038b233ba34779d. 2023-05-02 02:53:49,386 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1682996027177.408e891cab9b919f4038b233ba34779d.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf1/3663054878d64ef28b37a569207194d8.084b1607f7ed8615120a04e4729fbafa->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1/3663054878d64ef28b37a569207194d8-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf1/2cb812bd81a94565a0a1cf59a289bd05.084b1607f7ed8615120a04e4729fbafa->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1/2cb812bd81a94565a0a1cf59a289bd05-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf1/7285a69c46ad4b738dfaa96fd16f3ad9.084b1607f7ed8615120a04e4729fbafa->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1/7285a69c46ad4b738dfaa96fd16f3ad9-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf1/0a58882c16bc41b788b4deccd233becc.084b1607f7ed8615120a04e4729fbafa->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1/0a58882c16bc41b788b4deccd233becc-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf1/testSplitTableRegionUnevenDaughter=084b1607f7ed8615120a04e4729fbafa-1893934900b2482ea071cead41006e9d, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf1/testSplitTableRegionUnevenDaughter=084b1607f7ed8615120a04e4729fbafa-0cbb9fddcfd74f8c853f11e6ad27b8c1, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf1/testSplitTableRegionUnevenDaughter=084b1607f7ed8615120a04e4729fbafa-6d14eb71c6f244cba148da66de9514b6] to archive 2023-05-02 02:53:49,387 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1682996027177.408e891cab9b919f4038b233ba34779d.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-02 02:53:49,393 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1682996027177.408e891cab9b919f4038b233ba34779d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf1/3663054878d64ef28b37a569207194d8.084b1607f7ed8615120a04e4729fbafa to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf1/3663054878d64ef28b37a569207194d8.084b1607f7ed8615120a04e4729fbafa 2023-05-02 02:53:49,395 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1682996027177.408e891cab9b919f4038b233ba34779d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf1/2cb812bd81a94565a0a1cf59a289bd05.084b1607f7ed8615120a04e4729fbafa to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf1/2cb812bd81a94565a0a1cf59a289bd05.084b1607f7ed8615120a04e4729fbafa 2023-05-02 02:53:49,397 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1682996027177.408e891cab9b919f4038b233ba34779d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf1/7285a69c46ad4b738dfaa96fd16f3ad9.084b1607f7ed8615120a04e4729fbafa to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf1/7285a69c46ad4b738dfaa96fd16f3ad9.084b1607f7ed8615120a04e4729fbafa 2023-05-02 02:53:49,399 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1682996027177.408e891cab9b919f4038b233ba34779d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf1/0a58882c16bc41b788b4deccd233becc.084b1607f7ed8615120a04e4729fbafa to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf1/0a58882c16bc41b788b4deccd233becc.084b1607f7ed8615120a04e4729fbafa 2023-05-02 02:53:49,401 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1682996027177.408e891cab9b919f4038b233ba34779d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf1/testSplitTableRegionUnevenDaughter=084b1607f7ed8615120a04e4729fbafa-1893934900b2482ea071cead41006e9d to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf1/testSplitTableRegionUnevenDaughter=084b1607f7ed8615120a04e4729fbafa-1893934900b2482ea071cead41006e9d 2023-05-02 02:53:49,403 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1682996027177.408e891cab9b919f4038b233ba34779d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf1/testSplitTableRegionUnevenDaughter=084b1607f7ed8615120a04e4729fbafa-0cbb9fddcfd74f8c853f11e6ad27b8c1 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf1/testSplitTableRegionUnevenDaughter=084b1607f7ed8615120a04e4729fbafa-0cbb9fddcfd74f8c853f11e6ad27b8c1 2023-05-02 02:53:49,405 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1682996027177.408e891cab9b919f4038b233ba34779d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf1/testSplitTableRegionUnevenDaughter=084b1607f7ed8615120a04e4729fbafa-6d14eb71c6f244cba148da66de9514b6 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf1/testSplitTableRegionUnevenDaughter=084b1607f7ed8615120a04e4729fbafa-6d14eb71c6f244cba148da66de9514b6 2023-05-02 02:53:49,433 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1682996027177.408e891cab9b919f4038b233ba34779d.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf2/b5f2f094012f478e9aa22a132932427f.084b1607f7ed8615120a04e4729fbafa->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2/b5f2f094012f478e9aa22a132932427f-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf2/ea424b3182454ff5a1a8d6c3795602a2.084b1607f7ed8615120a04e4729fbafa->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2/ea424b3182454ff5a1a8d6c3795602a2-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf2/9cf56291619249dda5dc07d66464b1af.084b1607f7ed8615120a04e4729fbafa->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2/9cf56291619249dda5dc07d66464b1af-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf2/cf84b8983146452aa1df04c6b4dfbbb6.084b1607f7ed8615120a04e4729fbafa->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2/cf84b8983146452aa1df04c6b4dfbbb6-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf2/testSplitTableRegionUnevenDaughter=084b1607f7ed8615120a04e4729fbafa-8d86bdc24ade4df9963a5d3f077111d2, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf2/testSplitTableRegionUnevenDaughter=084b1607f7ed8615120a04e4729fbafa-7fcf6b93d24a4fb5bb305342e41fd972, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf2/testSplitTableRegionUnevenDaughter=084b1607f7ed8615120a04e4729fbafa-a68c693682044c6c8576e930cd4eaf62] to archive 2023-05-02 02:53:49,434 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1682996027177.408e891cab9b919f4038b233ba34779d.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-02 02:53:49,437 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1682996027177.408e891cab9b919f4038b233ba34779d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf2/b5f2f094012f478e9aa22a132932427f.084b1607f7ed8615120a04e4729fbafa to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf2/b5f2f094012f478e9aa22a132932427f.084b1607f7ed8615120a04e4729fbafa 2023-05-02 02:53:49,439 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1682996027177.408e891cab9b919f4038b233ba34779d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf2/ea424b3182454ff5a1a8d6c3795602a2.084b1607f7ed8615120a04e4729fbafa to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf2/ea424b3182454ff5a1a8d6c3795602a2.084b1607f7ed8615120a04e4729fbafa 2023-05-02 02:53:49,440 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1682996027177.408e891cab9b919f4038b233ba34779d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf2/9cf56291619249dda5dc07d66464b1af.084b1607f7ed8615120a04e4729fbafa to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf2/9cf56291619249dda5dc07d66464b1af.084b1607f7ed8615120a04e4729fbafa 2023-05-02 02:53:49,442 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1682996027177.408e891cab9b919f4038b233ba34779d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf2/cf84b8983146452aa1df04c6b4dfbbb6.084b1607f7ed8615120a04e4729fbafa to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf2/cf84b8983146452aa1df04c6b4dfbbb6.084b1607f7ed8615120a04e4729fbafa 2023-05-02 02:53:49,444 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1682996027177.408e891cab9b919f4038b233ba34779d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf2/testSplitTableRegionUnevenDaughter=084b1607f7ed8615120a04e4729fbafa-8d86bdc24ade4df9963a5d3f077111d2 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf2/testSplitTableRegionUnevenDaughter=084b1607f7ed8615120a04e4729fbafa-8d86bdc24ade4df9963a5d3f077111d2 2023-05-02 02:53:49,446 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1682996027177.408e891cab9b919f4038b233ba34779d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf2/testSplitTableRegionUnevenDaughter=084b1607f7ed8615120a04e4729fbafa-7fcf6b93d24a4fb5bb305342e41fd972 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf2/testSplitTableRegionUnevenDaughter=084b1607f7ed8615120a04e4729fbafa-7fcf6b93d24a4fb5bb305342e41fd972 2023-05-02 02:53:49,448 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1682996027177.408e891cab9b919f4038b233ba34779d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf2/testSplitTableRegionUnevenDaughter=084b1607f7ed8615120a04e4729fbafa-a68c693682044c6c8576e930cd4eaf62 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf2/testSplitTableRegionUnevenDaughter=084b1607f7ed8615120a04e4729fbafa-a68c693682044c6c8576e930cd4eaf62 2023-05-02 02:53:49,453 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-05-02 02:53:49,454 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-02 02:53:49,455 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionUnevenDaughter,26,1682996027177.408e891cab9b919f4038b233ba34779d. 2023-05-02 02:53:49,455 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 408e891cab9b919f4038b233ba34779d: 2023-05-02 02:53:49,457 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 408e891cab9b919f4038b233ba34779d 2023-05-02 02:53:49,457 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close ea7683cd3436a62eb16418a3c875d563 2023-05-02 02:53:49,457 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing ea7683cd3436a62eb16418a3c875d563, disabling compactions & flushes 2023-05-02 02:53:49,457 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionUnevenDaughter,,1682996027177.ea7683cd3436a62eb16418a3c875d563. 2023-05-02 02:53:49,457 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1682996027177.ea7683cd3436a62eb16418a3c875d563. 2023-05-02 02:53:49,457 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionUnevenDaughter,,1682996027177.ea7683cd3436a62eb16418a3c875d563. after waiting 0 ms 2023-05-02 02:53:49,458 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionUnevenDaughter,,1682996027177.ea7683cd3436a62eb16418a3c875d563. 2023-05-02 02:53:49,458 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=408e891cab9b919f4038b233ba34779d, regionState=CLOSED 2023-05-02 02:53:49,459 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,26,1682996027177.408e891cab9b919f4038b233ba34779d.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1682996029458"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682996029458"}]},"ts":"1682996029458"} 2023-05-02 02:53:49,459 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1682996027177.ea7683cd3436a62eb16418a3c875d563.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/ea7683cd3436a62eb16418a3c875d563/cf1/3663054878d64ef28b37a569207194d8.084b1607f7ed8615120a04e4729fbafa->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1/3663054878d64ef28b37a569207194d8-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/ea7683cd3436a62eb16418a3c875d563/cf1/2cb812bd81a94565a0a1cf59a289bd05.084b1607f7ed8615120a04e4729fbafa->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1/2cb812bd81a94565a0a1cf59a289bd05-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/ea7683cd3436a62eb16418a3c875d563/cf1/7285a69c46ad4b738dfaa96fd16f3ad9.084b1607f7ed8615120a04e4729fbafa->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1/7285a69c46ad4b738dfaa96fd16f3ad9-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/ea7683cd3436a62eb16418a3c875d563/cf1/0a58882c16bc41b788b4deccd233becc.084b1607f7ed8615120a04e4729fbafa->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1/0a58882c16bc41b788b4deccd233becc-bottom] to archive 2023-05-02 02:53:49,460 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1682996027177.ea7683cd3436a62eb16418a3c875d563.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-02 02:53:49,464 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1682996027177.ea7683cd3436a62eb16418a3c875d563.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/ea7683cd3436a62eb16418a3c875d563/cf1/3663054878d64ef28b37a569207194d8.084b1607f7ed8615120a04e4729fbafa to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionUnevenDaughter/ea7683cd3436a62eb16418a3c875d563/cf1/3663054878d64ef28b37a569207194d8.084b1607f7ed8615120a04e4729fbafa 2023-05-02 02:53:49,465 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=22, resume processing ppid=21 2023-05-02 02:53:49,465 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=22, ppid=21, state=SUCCESS; CloseRegionProcedure 408e891cab9b919f4038b233ba34779d, server=jenkins-hbase4.apache.org,38925,1682996021633 in 244 msec 2023-05-02 02:53:49,467 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1682996027177.ea7683cd3436a62eb16418a3c875d563.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/ea7683cd3436a62eb16418a3c875d563/cf1/2cb812bd81a94565a0a1cf59a289bd05.084b1607f7ed8615120a04e4729fbafa to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionUnevenDaughter/ea7683cd3436a62eb16418a3c875d563/cf1/2cb812bd81a94565a0a1cf59a289bd05.084b1607f7ed8615120a04e4729fbafa 2023-05-02 02:53:49,468 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=408e891cab9b919f4038b233ba34779d, UNASSIGN in 253 msec 2023-05-02 02:53:49,470 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1682996027177.ea7683cd3436a62eb16418a3c875d563.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/ea7683cd3436a62eb16418a3c875d563/cf1/7285a69c46ad4b738dfaa96fd16f3ad9.084b1607f7ed8615120a04e4729fbafa to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionUnevenDaughter/ea7683cd3436a62eb16418a3c875d563/cf1/7285a69c46ad4b738dfaa96fd16f3ad9.084b1607f7ed8615120a04e4729fbafa 2023-05-02 02:53:49,471 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1682996027177.ea7683cd3436a62eb16418a3c875d563.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/ea7683cd3436a62eb16418a3c875d563/cf1/0a58882c16bc41b788b4deccd233becc.084b1607f7ed8615120a04e4729fbafa to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionUnevenDaughter/ea7683cd3436a62eb16418a3c875d563/cf1/0a58882c16bc41b788b4deccd233becc.084b1607f7ed8615120a04e4729fbafa 2023-05-02 02:53:49,474 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1682996027177.ea7683cd3436a62eb16418a3c875d563.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/ea7683cd3436a62eb16418a3c875d563/cf2/b5f2f094012f478e9aa22a132932427f.084b1607f7ed8615120a04e4729fbafa->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2/b5f2f094012f478e9aa22a132932427f-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/ea7683cd3436a62eb16418a3c875d563/cf2/ea424b3182454ff5a1a8d6c3795602a2.084b1607f7ed8615120a04e4729fbafa->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2/ea424b3182454ff5a1a8d6c3795602a2-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/ea7683cd3436a62eb16418a3c875d563/cf2/9cf56291619249dda5dc07d66464b1af.084b1607f7ed8615120a04e4729fbafa->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2/9cf56291619249dda5dc07d66464b1af-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/ea7683cd3436a62eb16418a3c875d563/cf2/cf84b8983146452aa1df04c6b4dfbbb6.084b1607f7ed8615120a04e4729fbafa->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2/cf84b8983146452aa1df04c6b4dfbbb6-bottom] to archive 2023-05-02 02:53:49,475 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1682996027177.ea7683cd3436a62eb16418a3c875d563.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-02 02:53:49,477 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1682996027177.ea7683cd3436a62eb16418a3c875d563.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/ea7683cd3436a62eb16418a3c875d563/cf2/b5f2f094012f478e9aa22a132932427f.084b1607f7ed8615120a04e4729fbafa to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionUnevenDaughter/ea7683cd3436a62eb16418a3c875d563/cf2/b5f2f094012f478e9aa22a132932427f.084b1607f7ed8615120a04e4729fbafa 2023-05-02 02:53:49,478 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1682996027177.ea7683cd3436a62eb16418a3c875d563.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/ea7683cd3436a62eb16418a3c875d563/cf2/ea424b3182454ff5a1a8d6c3795602a2.084b1607f7ed8615120a04e4729fbafa to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionUnevenDaughter/ea7683cd3436a62eb16418a3c875d563/cf2/ea424b3182454ff5a1a8d6c3795602a2.084b1607f7ed8615120a04e4729fbafa 2023-05-02 02:53:49,480 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1682996027177.ea7683cd3436a62eb16418a3c875d563.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/ea7683cd3436a62eb16418a3c875d563/cf2/9cf56291619249dda5dc07d66464b1af.084b1607f7ed8615120a04e4729fbafa to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionUnevenDaughter/ea7683cd3436a62eb16418a3c875d563/cf2/9cf56291619249dda5dc07d66464b1af.084b1607f7ed8615120a04e4729fbafa 2023-05-02 02:53:49,481 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1682996027177.ea7683cd3436a62eb16418a3c875d563.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/ea7683cd3436a62eb16418a3c875d563/cf2/cf84b8983146452aa1df04c6b4dfbbb6.084b1607f7ed8615120a04e4729fbafa to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionUnevenDaughter/ea7683cd3436a62eb16418a3c875d563/cf2/cf84b8983146452aa1df04c6b4dfbbb6.084b1607f7ed8615120a04e4729fbafa 2023-05-02 02:53:49,486 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionUnevenDaughter/ea7683cd3436a62eb16418a3c875d563/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-05-02 02:53:49,487 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-02 02:53:49,489 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionUnevenDaughter,,1682996027177.ea7683cd3436a62eb16418a3c875d563. 2023-05-02 02:53:49,489 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for ea7683cd3436a62eb16418a3c875d563: 2023-05-02 02:53:49,491 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed ea7683cd3436a62eb16418a3c875d563 2023-05-02 02:53:49,491 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=ea7683cd3436a62eb16418a3c875d563, regionState=CLOSED 2023-05-02 02:53:49,492 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1682996027177.ea7683cd3436a62eb16418a3c875d563.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1682996029491"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682996029491"}]},"ts":"1682996029491"} 2023-05-02 02:53:49,496 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=23, resume processing ppid=20 2023-05-02 02:53:49,496 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=23, ppid=20, state=SUCCESS; CloseRegionProcedure ea7683cd3436a62eb16418a3c875d563, server=jenkins-hbase4.apache.org,38925,1682996021633 in 269 msec 2023-05-02 02:53:49,499 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=20, resume processing ppid=19 2023-05-02 02:53:49,499 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=ea7683cd3436a62eb16418a3c875d563, UNASSIGN in 284 msec 2023-05-02 02:53:49,500 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682996029500"}]},"ts":"1682996029500"} 2023-05-02 02:53:49,502 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLED in hbase:meta 2023-05-02 02:53:49,504 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionUnevenDaughter to state=DISABLED 2023-05-02 02:53:49,507 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter in 318 msec 2023-05-02 02:53:49,514 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-05-02 02:53:49,515 INFO [Listener at localhost/37119] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 19 completed 2023-05-02 02:53:49,522 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testSplitTableRegionUnevenDaughter 2023-05-02 02:53:49,530 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure2.ProcedureExecutor(1029): Stored pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:53:49,533 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:53:49,535 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=24, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:53:49,537 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-05-02 02:53:49,545 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa 2023-05-02 02:53:49,545 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionUnevenDaughter/ea7683cd3436a62eb16418a3c875d563 2023-05-02 02:53:49,545 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d 2023-05-02 02:53:49,549 DEBUG [HFileArchiver-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf1, FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf2, FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/recovered.edits] 2023-05-02 02:53:49,549 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1, FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2, FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/recovered.edits] 2023-05-02 02:53:49,549 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionUnevenDaughter/ea7683cd3436a62eb16418a3c875d563/cf1, FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionUnevenDaughter/ea7683cd3436a62eb16418a3c875d563/cf2, FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionUnevenDaughter/ea7683cd3436a62eb16418a3c875d563/recovered.edits] 2023-05-02 02:53:49,554 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf1/0da696f482c841fc85dde2d9dde1afed to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf1/0da696f482c841fc85dde2d9dde1afed 2023-05-02 02:53:49,555 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1/0a58882c16bc41b788b4deccd233becc to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1/0a58882c16bc41b788b4deccd233becc 2023-05-02 02:53:49,555 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionUnevenDaughter/ea7683cd3436a62eb16418a3c875d563/cf1/fd2597bbde7e4768b1f8e7bae14ba9af to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionUnevenDaughter/ea7683cd3436a62eb16418a3c875d563/cf1/fd2597bbde7e4768b1f8e7bae14ba9af 2023-05-02 02:53:49,557 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1/0cbb9fddcfd74f8c853f11e6ad27b8c1 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1/0cbb9fddcfd74f8c853f11e6ad27b8c1 2023-05-02 02:53:49,559 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf2/cdd2f69c524441fe935977abb83c9c50 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/cf2/cdd2f69c524441fe935977abb83c9c50 2023-05-02 02:53:49,559 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionUnevenDaughter/ea7683cd3436a62eb16418a3c875d563/cf2/fd29bac7a8e14d8190522b2883a65b5f to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionUnevenDaughter/ea7683cd3436a62eb16418a3c875d563/cf2/fd29bac7a8e14d8190522b2883a65b5f 2023-05-02 02:53:49,560 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1/1893934900b2482ea071cead41006e9d to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1/1893934900b2482ea071cead41006e9d 2023-05-02 02:53:49,562 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1/2cb812bd81a94565a0a1cf59a289bd05 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1/2cb812bd81a94565a0a1cf59a289bd05 2023-05-02 02:53:49,563 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/recovered.edits/90.seqid to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d/recovered.edits/90.seqid 2023-05-02 02:53:49,564 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionUnevenDaughter/ea7683cd3436a62eb16418a3c875d563/recovered.edits/90.seqid to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionUnevenDaughter/ea7683cd3436a62eb16418a3c875d563/recovered.edits/90.seqid 2023-05-02 02:53:49,565 DEBUG [HFileArchiver-5] backup.HFileArchiver(596): Deleted hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionUnevenDaughter/408e891cab9b919f4038b233ba34779d 2023-05-02 02:53:49,565 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1/3663054878d64ef28b37a569207194d8 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1/3663054878d64ef28b37a569207194d8 2023-05-02 02:53:49,565 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionUnevenDaughter/ea7683cd3436a62eb16418a3c875d563 2023-05-02 02:53:49,566 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1/6d14eb71c6f244cba148da66de9514b6 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1/6d14eb71c6f244cba148da66de9514b6 2023-05-02 02:53:49,568 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1/7285a69c46ad4b738dfaa96fd16f3ad9 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf1/7285a69c46ad4b738dfaa96fd16f3ad9 2023-05-02 02:53:49,571 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2/7fcf6b93d24a4fb5bb305342e41fd972 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2/7fcf6b93d24a4fb5bb305342e41fd972 2023-05-02 02:53:49,573 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2/8d86bdc24ade4df9963a5d3f077111d2 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2/8d86bdc24ade4df9963a5d3f077111d2 2023-05-02 02:53:49,575 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2/9cf56291619249dda5dc07d66464b1af to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2/9cf56291619249dda5dc07d66464b1af 2023-05-02 02:53:49,576 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2/a68c693682044c6c8576e930cd4eaf62 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2/a68c693682044c6c8576e930cd4eaf62 2023-05-02 02:53:49,578 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2/b5f2f094012f478e9aa22a132932427f to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2/b5f2f094012f478e9aa22a132932427f 2023-05-02 02:53:49,579 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2/cf84b8983146452aa1df04c6b4dfbbb6 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2/cf84b8983146452aa1df04c6b4dfbbb6 2023-05-02 02:53:49,581 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2/ea424b3182454ff5a1a8d6c3795602a2 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/cf2/ea424b3182454ff5a1a8d6c3795602a2 2023-05-02 02:53:49,584 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/recovered.edits/85.seqid to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa/recovered.edits/85.seqid 2023-05-02 02:53:49,585 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionUnevenDaughter/084b1607f7ed8615120a04e4729fbafa 2023-05-02 02:53:49,585 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-05-02 02:53:49,588 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=24, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:53:49,611 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionUnevenDaughter from hbase:meta 2023-05-02 02:53:49,639 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-05-02 02:53:49,655 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionUnevenDaughter' descriptor. 2023-05-02 02:53:49,657 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=24, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:53:49,657 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionUnevenDaughter' from region states. 2023-05-02 02:53:49,657 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1682996025014.084b1607f7ed8615120a04e4729fbafa.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1682996029657"}]},"ts":"9223372036854775807"} 2023-05-02 02:53:49,658 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1682996027177.ea7683cd3436a62eb16418a3c875d563.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1682996029657"}]},"ts":"9223372036854775807"} 2023-05-02 02:53:49,658 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,26,1682996027177.408e891cab9b919f4038b233ba34779d.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1682996029657"}]},"ts":"9223372036854775807"} 2023-05-02 02:53:49,661 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-05-02 02:53:49,661 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 084b1607f7ed8615120a04e4729fbafa, NAME => 'testSplitTableRegionUnevenDaughter,,1682996025014.084b1607f7ed8615120a04e4729fbafa.', STARTKEY => '', ENDKEY => ''}, {ENCODED => ea7683cd3436a62eb16418a3c875d563, NAME => 'testSplitTableRegionUnevenDaughter,,1682996027177.ea7683cd3436a62eb16418a3c875d563.', STARTKEY => '', ENDKEY => '26'}, {ENCODED => 408e891cab9b919f4038b233ba34779d, NAME => 'testSplitTableRegionUnevenDaughter,26,1682996027177.408e891cab9b919f4038b233ba34779d.', STARTKEY => '26', ENDKEY => ''}] 2023-05-02 02:53:49,661 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionUnevenDaughter' as deleted. 2023-05-02 02:53:49,661 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1682996029661"}]},"ts":"9223372036854775807"} 2023-05-02 02:53:49,663 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionUnevenDaughter state from META 2023-05-02 02:53:49,666 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=24, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:53:49,668 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter in 143 msec 2023-05-02 02:53:49,840 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-05-02 02:53:49,841 INFO [Listener at localhost/37119] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 24 completed 2023-05-02 02:53:49,858 INFO [Listener at localhost/37119] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=390 (was 358) Potentially hanging thread: RSProcedureDispatcher-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1210764838_15 at /127.0.0.1:53066 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1210764838_15 at /127.0.0.1:52994 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:1;jenkins-hbase4:38925-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2c67b29-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1210764838_15 at /127.0.0.1:34200 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1210764838_15 at /127.0.0.1:38812 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: hconnection-0x1f586e56-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1210764838_15 at /127.0.0.1:34468 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2c67b29-metaLookup-shared--pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1210764838_15 at /127.0.0.1:53018 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x1f586e56-shared-pool-9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x1f586e56-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x1f586e56-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1210764838_15 at /127.0.0.1:38470 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1210764838_15 at /127.0.0.1:34170 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1210764838_15 at /127.0.0.1:38814 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1210764838_15 at /127.0.0.1:53300 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1210764838_15 at /127.0.0.1:38512 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2c67b29-metaLookup-shared--pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x1f586e56-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1210764838_15 at /127.0.0.1:53064 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2c67b29-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1210764838_15 at /127.0.0.1:53020 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: (jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2c67b29-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1210764838_15 at /127.0.0.1:38546 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1210764838_15 at /127.0.0.1:53004 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2c67b29-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1210764838_15 at /127.0.0.1:38584 [Waiting for operation #17] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=842 (was 692) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=306 (was 315), ProcessCount=170 (was 170), AvailableMemoryMB=13974 (was 14048) 2023-05-02 02:53:49,872 INFO [Listener at localhost/37119] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=390, OpenFileDescriptor=842, MaxFileDescriptor=60000, SystemLoadAverage=306, ProcessCount=170, AvailableMemoryMB=13974 2023-05-02 02:53:49,872 WARN [Listener at localhost/37119] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-02 02:53:49,878 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-05-02 02:53:49,879 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-05-02 02:53:49,881 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(1029): Stored pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitWithoutPONR 2023-05-02 02:53:49,882 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_PRE_OPERATION 2023-05-02 02:53:49,883 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-05-02 02:53:49,885 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:49,886 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855 empty. 2023-05-02 02:53:49,887 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:49,887 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-05-02 02:53:49,902 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/.tabledesc/.tableinfo.0000000001 2023-05-02 02:53:49,903 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(7675): creating {ENCODED => b92c1c7e8ba3ec18a0b4e94867da9855, NAME => 'testSplitWithoutPONR,,1682996029879.b92c1c7e8ba3ec18a0b4e94867da9855.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitWithoutPONR', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp 2023-05-02 02:53:49,915 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(866): Instantiated testSplitWithoutPONR,,1682996029879.b92c1c7e8ba3ec18a0b4e94867da9855.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-02 02:53:49,915 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1604): Closing b92c1c7e8ba3ec18a0b4e94867da9855, disabling compactions & flushes 2023-05-02 02:53:49,915 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1626): Closing region testSplitWithoutPONR,,1682996029879.b92c1c7e8ba3ec18a0b4e94867da9855. 2023-05-02 02:53:49,915 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitWithoutPONR,,1682996029879.b92c1c7e8ba3ec18a0b4e94867da9855. 2023-05-02 02:53:49,915 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitWithoutPONR,,1682996029879.b92c1c7e8ba3ec18a0b4e94867da9855. after waiting 0 ms 2023-05-02 02:53:49,915 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitWithoutPONR,,1682996029879.b92c1c7e8ba3ec18a0b4e94867da9855. 2023-05-02 02:53:49,915 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1838): Closed testSplitWithoutPONR,,1682996029879.b92c1c7e8ba3ec18a0b4e94867da9855. 2023-05-02 02:53:49,915 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1558): Region close journal for b92c1c7e8ba3ec18a0b4e94867da9855: 2023-05-02 02:53:49,918 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ADD_TO_META 2023-05-02 02:53:49,919 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1682996029879.b92c1c7e8ba3ec18a0b4e94867da9855.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1682996029919"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682996029919"}]},"ts":"1682996029919"} 2023-05-02 02:53:49,921 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-05-02 02:53:49,922 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-05-02 02:53:49,922 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682996029922"}]},"ts":"1682996029922"} 2023-05-02 02:53:49,924 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLING in hbase:meta 2023-05-02 02:53:49,927 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-05-02 02:53:49,928 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-05-02 02:53:49,928 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-05-02 02:53:49,928 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-05-02 02:53:49,928 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-05-02 02:53:49,929 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=b92c1c7e8ba3ec18a0b4e94867da9855, ASSIGN}] 2023-05-02 02:53:49,931 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=b92c1c7e8ba3ec18a0b4e94867da9855, ASSIGN 2023-05-02 02:53:49,932 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=b92c1c7e8ba3ec18a0b4e94867da9855, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42019,1682996021676; forceNewPlan=false, retain=false 2023-05-02 02:53:50,082 INFO [jenkins-hbase4:41119] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-05-02 02:53:50,083 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=b92c1c7e8ba3ec18a0b4e94867da9855, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:50,083 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1682996029879.b92c1c7e8ba3ec18a0b4e94867da9855.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1682996030083"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1682996030083"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682996030083"}]},"ts":"1682996030083"} 2023-05-02 02:53:50,086 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=27, ppid=26, state=RUNNABLE; OpenRegionProcedure b92c1c7e8ba3ec18a0b4e94867da9855, server=jenkins-hbase4.apache.org,42019,1682996021676}] 2023-05-02 02:53:50,242 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitWithoutPONR,,1682996029879.b92c1c7e8ba3ec18a0b4e94867da9855. 2023-05-02 02:53:50,243 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => b92c1c7e8ba3ec18a0b4e94867da9855, NAME => 'testSplitWithoutPONR,,1682996029879.b92c1c7e8ba3ec18a0b4e94867da9855.', STARTKEY => '', ENDKEY => ''} 2023-05-02 02:53:50,243 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-02 02:53:50,243 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:50,243 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitWithoutPONR,,1682996029879.b92c1c7e8ba3ec18a0b4e94867da9855.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-02 02:53:50,243 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:50,243 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:50,245 INFO [StoreOpener-b92c1c7e8ba3ec18a0b4e94867da9855-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:50,246 DEBUG [StoreOpener-b92c1c7e8ba3ec18a0b4e94867da9855-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1 2023-05-02 02:53:50,246 DEBUG [StoreOpener-b92c1c7e8ba3ec18a0b4e94867da9855-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1 2023-05-02 02:53:50,247 INFO [StoreOpener-b92c1c7e8ba3ec18a0b4e94867da9855-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b92c1c7e8ba3ec18a0b4e94867da9855 columnFamilyName cf1 2023-05-02 02:53:50,247 INFO [StoreOpener-b92c1c7e8ba3ec18a0b4e94867da9855-1] regionserver.HStore(310): Store=b92c1c7e8ba3ec18a0b4e94867da9855/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-02 02:53:50,247 INFO [StoreOpener-b92c1c7e8ba3ec18a0b4e94867da9855-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:50,249 DEBUG [StoreOpener-b92c1c7e8ba3ec18a0b4e94867da9855-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2 2023-05-02 02:53:50,249 DEBUG [StoreOpener-b92c1c7e8ba3ec18a0b4e94867da9855-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2 2023-05-02 02:53:50,251 INFO [StoreOpener-b92c1c7e8ba3ec18a0b4e94867da9855-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b92c1c7e8ba3ec18a0b4e94867da9855 columnFamilyName cf2 2023-05-02 02:53:50,251 INFO [StoreOpener-b92c1c7e8ba3ec18a0b4e94867da9855-1] regionserver.HStore(310): Store=b92c1c7e8ba3ec18a0b4e94867da9855/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-02 02:53:50,252 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:50,253 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:50,255 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-02 02:53:50,256 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:50,259 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-05-02 02:53:50,259 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened b92c1c7e8ba3ec18a0b4e94867da9855; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63305153, jitterRate=-0.05667971074581146}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-02 02:53:50,259 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for b92c1c7e8ba3ec18a0b4e94867da9855: 2023-05-02 02:53:50,261 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,,1682996029879.b92c1c7e8ba3ec18a0b4e94867da9855., pid=27, masterSystemTime=1682996030238 2023-05-02 02:53:50,263 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,,1682996029879.b92c1c7e8ba3ec18a0b4e94867da9855. 2023-05-02 02:53:50,263 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitWithoutPONR,,1682996029879.b92c1c7e8ba3ec18a0b4e94867da9855. 2023-05-02 02:53:50,264 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=b92c1c7e8ba3ec18a0b4e94867da9855, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:50,264 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1682996029879.b92c1c7e8ba3ec18a0b4e94867da9855.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1682996030263"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1682996030263"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1682996030263"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682996030263"}]},"ts":"1682996030263"} 2023-05-02 02:53:50,269 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=27, resume processing ppid=26 2023-05-02 02:53:50,269 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=27, ppid=26, state=SUCCESS; OpenRegionProcedure b92c1c7e8ba3ec18a0b4e94867da9855, server=jenkins-hbase4.apache.org,42019,1682996021676 in 180 msec 2023-05-02 02:53:50,272 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=26, resume processing ppid=25 2023-05-02 02:53:50,272 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=26, ppid=25, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=b92c1c7e8ba3ec18a0b4e94867da9855, ASSIGN in 340 msec 2023-05-02 02:53:50,273 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-05-02 02:53:50,273 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682996030273"}]},"ts":"1682996030273"} 2023-05-02 02:53:50,274 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLED in hbase:meta 2023-05-02 02:53:50,277 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_POST_OPERATION 2023-05-02 02:53:50,278 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR in 398 msec 2023-05-02 02:53:50,385 DEBUG [Listener at localhost/37119] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-05-02 02:53:50,389 INFO [RS-EventLoopGroup-5-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58782, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-05-02 02:53:50,397 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-02 02:53:50,400 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-05-02 02:53:50,400 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-05-02 02:53:50,400 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:50,401 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-05-02 02:53:50,401 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-05-02 02:53:50,401 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:50,401 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-02 02:53:50,403 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:50,403 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:50,403 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:50,403 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:50,403 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:50,403 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:50,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:50,403 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:50,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:50,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:50,403 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:50,403 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-02 02:53:50,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-02 02:53:50,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-02 02:53:50,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-02 02:53:50,404 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:50,404 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:50,409 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:50,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:53:50,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-02 02:53:50,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:53:50,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:53:50,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-02 02:53:50,409 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-02 02:53:50,409 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-02 02:53:50,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-02 02:53:50,410 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-02 02:53:50,410 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-02 02:53:50,410 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-02 02:53:50,410 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-02 02:53:50,410 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-02 02:53:50,410 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:50,410 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-02 02:53:50,411 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-02 02:53:50,411 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-02 02:53:50,411 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:50,410 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:50,411 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-02 02:53:50,412 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-02 02:53:50,412 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-02 02:53:50,412 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-02 02:53:50,412 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-02 02:53:50,412 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-02 02:53:50,412 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1682996029879.b92c1c7e8ba3ec18a0b4e94867da9855. 2023-05-02 02:53:50,412 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38925,1682996021633' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-02 02:53:50,413 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1682996029879.b92c1c7e8ba3ec18a0b4e94867da9855. started... 2023-05-02 02:53:50,412 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-02 02:53:50,413 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b92c1c7e8ba3ec18a0b4e94867da9855 2/2 column families, dataSize=120 B heapSize=928 B 2023-05-02 02:53:50,414 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:50,416 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:50,416 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:50,416 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:53:50,416 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:50,416 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:53:50,427 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/.tmp/cf1/5317f0b5b225487c82a9f79b071b429d 2023-05-02 02:53:50,452 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/.tmp/cf2/b94fd300967f43f7a6ca454238517875 2023-05-02 02:53:50,458 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/.tmp/cf1/5317f0b5b225487c82a9f79b071b429d as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/5317f0b5b225487c82a9f79b071b429d 2023-05-02 02:53:50,464 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/5317f0b5b225487c82a9f79b071b429d, entries=2, sequenceid=6, filesize=4.8 K 2023-05-02 02:53:50,465 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/.tmp/cf2/b94fd300967f43f7a6ca454238517875 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/b94fd300967f43f7a6ca454238517875 2023-05-02 02:53:50,471 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/b94fd300967f43f7a6ca454238517875, entries=2, sequenceid=6, filesize=4.8 K 2023-05-02 02:53:50,472 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for b92c1c7e8ba3ec18a0b4e94867da9855 in 59ms, sequenceid=6, compaction requested=false 2023-05-02 02:53:50,473 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitWithoutPONR' 2023-05-02 02:53:50,473 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b92c1c7e8ba3ec18a0b4e94867da9855: 2023-05-02 02:53:50,473 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1682996029879.b92c1c7e8ba3ec18a0b4e94867da9855. 2023-05-02 02:53:50,473 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-02 02:53:50,474 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-02 02:53:50,474 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:50,474 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-02 02:53:50,474 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42019,1682996021676' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-02 02:53:50,475 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:50,475 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:50,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:50,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:53:50,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:53:50,476 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:50,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:53:50,476 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:53:50,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:53:50,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-02 02:53:50,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:50,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:50,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:50,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:53:50,478 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42019,1682996021676' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-05-02 02:53:50,478 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@320dfb0[Count = 0] remaining members to acquire global barrier 2023-05-02 02:53:50,478 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-05-02 02:53:50,479 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:50,480 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:50,480 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:50,481 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:50,481 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:50,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:50,481 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:50,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:50,481 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-02 02:53:50,481 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-02 02:53:50,481 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-02 02:53:50,481 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-02 02:53:50,481 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:50,481 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,42019,1682996021676' in zk 2023-05-02 02:53:50,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:50,481 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,38925,1682996021633' in zk 2023-05-02 02:53:50,481 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-02 02:53:50,481 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-02 02:53:50,481 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,39081,1682996021579' in zk 2023-05-02 02:53:50,481 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:50,482 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-02 02:53:50,482 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:50,483 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-02 02:53:50,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:50,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:53:50,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:53:50,483 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:50,483 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-02 02:53:50,484 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-02 02:53:50,484 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-02 02:53:50,484 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:50,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:53:50,484 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-02 02:53:50,484 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:50,485 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-02 02:53:50,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:53:50,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-02 02:53:50,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:50,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:50,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:50,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:53:50,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-02 02:53:50,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:50,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:50,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:50,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,42019,1682996021676': 2023-05-02 02:53:50,490 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42019,1682996021676' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-05-02 02:53:50,490 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-05-02 02:53:50,490 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-02 02:53:50,490 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-02 02:53:50,490 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-05-02 02:53:50,490 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-02 02:53:50,491 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:50,491 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:50,492 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:50,492 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:50,492 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:50,492 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:50,492 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:50,492 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:50,492 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:50,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:50,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:53:50,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:53:50,492 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:50,492 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:50,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:50,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:50,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:50,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:53:50,493 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:50,493 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:50,493 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:50,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:50,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:50,493 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:50,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:50,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-02 02:53:50,493 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:50,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:50,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:50,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:50,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:53:50,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-02 02:53:50,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:50,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:50,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:50,495 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:50,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:53:50,496 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:50,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-02 02:53:50,496 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:50,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:50,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:50,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:50,499 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:50,499 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:50,499 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-02 02:53:50,499 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:50,499 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:50,499 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:50,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:50,499 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:50,499 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:50,499 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:50,499 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-02 02:53:50,499 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:50,499 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:50,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:50,500 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:50,499 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:50,499 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:50,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:50,500 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:50,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:50,500 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:50,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:50,500 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:50,500 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-05-02 02:53:50,500 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-02 02:53:50,500 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:50,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:50,501 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-05-02 02:53:50,500 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:50,501 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-02 02:53:50,500 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:50,502 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:50,502 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:50,502 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:50,584 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-05-02 02:53:50,585 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-05-02 02:53:50,586 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-02 02:53:50,586 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro Metrics about HBase RegionObservers 2023-05-02 02:53:50,601 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-02 02:53:50,602 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-02 02:53:50,622 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-02 02:53:50,624 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-05-02 02:53:50,626 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-05-02 02:53:50,626 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-05-02 02:53:50,626 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:50,626 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-05-02 02:53:50,627 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-05-02 02:53:50,627 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:50,627 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-02 02:53:50,628 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:50,628 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:50,628 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:50,628 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:50,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:50,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:50,628 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:50,628 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:50,629 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:50,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:50,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-02 02:53:50,629 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:50,629 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-02 02:53:50,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-02 02:53:50,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-02 02:53:50,629 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:50,629 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:50,629 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:50,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:53:50,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-02 02:53:50,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:53:50,630 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-02 02:53:50,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:53:50,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-02 02:53:50,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-02 02:53:50,630 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-02 02:53:50,630 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-05-02 02:53:50,630 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-02 02:53:50,630 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-02 02:53:50,630 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-05-02 02:53:50,630 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-02 02:53:50,630 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:50,630 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-02 02:53:50,631 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-02 02:53:50,631 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-05-02 02:53:50,631 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-02 02:53:50,631 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-02 02:53:50,631 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38925,1682996021633' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-02 02:53:50,631 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-02 02:53:50,631 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:50,631 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-02 02:53:50,632 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:50,632 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-02 02:53:50,632 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-02 02:53:50,633 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-02 02:53:50,633 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-02 02:53:50,633 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-02 02:53:50,633 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-02 02:53:50,633 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1682996029879.b92c1c7e8ba3ec18a0b4e94867da9855. 2023-05-02 02:53:50,633 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1682996029879.b92c1c7e8ba3ec18a0b4e94867da9855. started... 2023-05-02 02:53:50,633 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b92c1c7e8ba3ec18a0b4e94867da9855 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-02 02:53:50,635 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:50,636 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:50,636 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:50,636 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:53:50,636 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:50,636 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:53:50,647 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/.tmp/cf1/a78695d99c44472fb73a8df5d283aed5 2023-05-02 02:53:50,663 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/.tmp/cf2/8f0768252daf4fca8efafdcc6273b3c6 2023-05-02 02:53:50,669 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/.tmp/cf1/a78695d99c44472fb73a8df5d283aed5 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/a78695d99c44472fb73a8df5d283aed5 2023-05-02 02:53:50,675 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/a78695d99c44472fb73a8df5d283aed5, entries=10, sequenceid=19, filesize=5.0 K 2023-05-02 02:53:50,676 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/.tmp/cf2/8f0768252daf4fca8efafdcc6273b3c6 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/8f0768252daf4fca8efafdcc6273b3c6 2023-05-02 02:53:50,681 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/8f0768252daf4fca8efafdcc6273b3c6, entries=10, sequenceid=19, filesize=5.0 K 2023-05-02 02:53:50,682 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for b92c1c7e8ba3ec18a0b4e94867da9855 in 49ms, sequenceid=19, compaction requested=false 2023-05-02 02:53:50,682 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b92c1c7e8ba3ec18a0b4e94867da9855: 2023-05-02 02:53:50,682 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1682996029879.b92c1c7e8ba3ec18a0b4e94867da9855. 2023-05-02 02:53:50,682 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-02 02:53:50,682 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-02 02:53:50,682 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:50,682 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-02 02:53:50,682 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42019,1682996021676' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-02 02:53:50,684 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:50,684 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:50,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:50,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:53:50,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:53:50,684 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:50,684 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:53:50,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:53:50,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:53:50,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-02 02:53:50,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:50,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:50,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:50,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:53:50,686 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42019,1682996021676' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-05-02 02:53:50,686 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7ae73005[Count = 0] remaining members to acquire global barrier 2023-05-02 02:53:50,686 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-05-02 02:53:50,686 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:50,688 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:50,688 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:50,688 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:50,688 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:50,688 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:50,688 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:50,688 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-02 02:53:50,688 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:50,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:50,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:50,688 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-02 02:53:50,688 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-02 02:53:50,688 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-02 02:53:50,688 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,39081,1682996021579' in zk 2023-05-02 02:53:50,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:50,688 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-02 02:53:50,688 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-02 02:53:50,688 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-02 02:53:50,688 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,38925,1682996021633' in zk 2023-05-02 02:53:50,688 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,42019,1682996021676' in zk 2023-05-02 02:53:50,690 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-02 02:53:50,690 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:50,690 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-02 02:53:50,694 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-02 02:53:50,694 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:50,694 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-02 02:53:50,695 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:50,695 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-02 02:53:50,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:50,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:53:50,695 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:50,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:53:50,695 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-02 02:53:50,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:53:50,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:53:50,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-02 02:53:50,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:50,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:50,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:50,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:53:50,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-02 02:53:50,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:50,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:50,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:50,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,42019,1682996021676': 2023-05-02 02:53:50,700 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42019,1682996021676' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-05-02 02:53:50,700 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-05-02 02:53:50,700 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-02 02:53:50,700 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-02 02:53:50,700 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-05-02 02:53:50,700 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-02 02:53:50,701 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:50,701 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:50,702 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:50,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:50,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:50,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:53:50,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:53:50,702 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:50,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:50,702 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:50,702 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:50,702 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:50,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:50,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:50,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:53:50,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:50,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:50,702 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:50,703 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:50,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:50,703 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:50,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:50,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:50,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-02 02:53:50,703 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:50,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:50,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:50,712 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:50,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:50,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:53:50,712 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:50,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-02 02:53:50,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:50,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:50,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:50,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:53:50,714 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:50,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-02 02:53:50,714 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:50,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:50,715 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:50,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:50,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:50,722 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:50,722 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-02 02:53:50,722 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:50,722 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:50,722 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:50,722 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:50,722 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:50,722 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:50,722 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-05-02 02:53:50,722 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:50,722 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-02 02:53:50,722 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:50,722 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:50,722 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:50,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:50,723 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-05-02 02:53:50,723 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:50,722 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:50,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:50,722 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:50,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:50,723 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-02 02:53:50,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:50,722 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-02 02:53:50,723 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:50,723 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:50,723 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:50,723 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:50,723 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:50,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:50,723 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:50,724 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:50,724 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:50,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:50,823 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-02 02:53:50,825 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-02 02:53:50,843 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-02 02:53:50,846 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-05-02 02:53:50,846 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-05-02 02:53:50,846 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-05-02 02:53:50,846 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:50,847 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-05-02 02:53:50,847 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-05-02 02:53:50,847 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:50,847 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-02 02:53:50,848 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:50,848 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:50,848 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:50,848 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:50,848 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:50,849 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:50,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:50,849 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:50,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:50,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:50,849 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:50,849 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-02 02:53:50,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-02 02:53:50,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-02 02:53:50,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-02 02:53:50,849 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:50,849 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:50,849 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:50,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:53:50,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:53:50,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-02 02:53:50,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-02 02:53:50,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:53:50,850 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-02 02:53:50,850 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-02 02:53:50,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-02 02:53:50,850 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-02 02:53:50,850 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-05-02 02:53:50,850 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-02 02:53:50,850 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-05-02 02:53:50,850 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-02 02:53:50,851 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-05-02 02:53:50,851 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:50,851 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-02 02:53:50,851 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-02 02:53:50,851 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-02 02:53:50,852 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:50,850 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-02 02:53:50,853 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-02 02:53:50,853 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-02 02:53:50,853 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1682996029879.b92c1c7e8ba3ec18a0b4e94867da9855. 2023-05-02 02:53:50,853 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-02 02:53:50,853 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-02 02:53:50,853 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-02 02:53:50,853 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-02 02:53:50,853 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:50,853 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1682996029879.b92c1c7e8ba3ec18a0b4e94867da9855. started... 2023-05-02 02:53:50,853 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-02 02:53:50,854 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-02 02:53:50,854 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38925,1682996021633' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-02 02:53:50,854 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b92c1c7e8ba3ec18a0b4e94867da9855 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-02 02:53:50,855 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:50,855 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:50,855 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:50,855 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:53:50,856 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:50,856 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:53:50,865 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/.tmp/cf1/de0cd278f5ae475c8e64f02e18179538 2023-05-02 02:53:50,887 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/.tmp/cf2/5c6252080bbc48e6b6a023e08c36b716 2023-05-02 02:53:50,893 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/.tmp/cf1/de0cd278f5ae475c8e64f02e18179538 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/de0cd278f5ae475c8e64f02e18179538 2023-05-02 02:53:50,898 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/de0cd278f5ae475c8e64f02e18179538, entries=10, sequenceid=32, filesize=5.0 K 2023-05-02 02:53:50,898 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/.tmp/cf2/5c6252080bbc48e6b6a023e08c36b716 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/5c6252080bbc48e6b6a023e08c36b716 2023-05-02 02:53:50,904 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/5c6252080bbc48e6b6a023e08c36b716, entries=10, sequenceid=32, filesize=5.0 K 2023-05-02 02:53:50,905 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for b92c1c7e8ba3ec18a0b4e94867da9855 in 51ms, sequenceid=32, compaction requested=true 2023-05-02 02:53:50,905 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b92c1c7e8ba3ec18a0b4e94867da9855: 2023-05-02 02:53:50,905 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1682996029879.b92c1c7e8ba3ec18a0b4e94867da9855. 2023-05-02 02:53:50,905 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-02 02:53:50,905 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-02 02:53:50,905 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:50,905 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-02 02:53:50,905 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42019,1682996021676' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-02 02:53:50,908 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:50,908 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:50,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:50,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:53:50,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:53:50,908 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:50,908 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:53:50,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:53:50,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:53:50,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-02 02:53:50,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:50,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:50,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:50,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:53:50,910 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42019,1682996021676' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-05-02 02:53:50,910 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@49490b1c[Count = 0] remaining members to acquire global barrier 2023-05-02 02:53:50,910 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-05-02 02:53:50,910 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:50,911 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:50,911 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:50,911 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:50,911 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:50,911 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:50,912 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:50,912 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:50,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:50,912 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-02 02:53:50,912 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-02 02:53:50,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:50,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:50,912 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-02 02:53:50,912 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-02 02:53:50,912 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-02 02:53:50,912 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,39081,1682996021579' in zk 2023-05-02 02:53:50,912 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-02 02:53:50,912 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-02 02:53:50,912 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,38925,1682996021633' in zk 2023-05-02 02:53:50,912 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,42019,1682996021676' in zk 2023-05-02 02:53:50,913 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-02 02:53:50,913 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:50,913 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-02 02:53:50,914 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:50,914 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-02 02:53:50,914 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-02 02:53:50,914 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:50,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:50,914 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-02 02:53:50,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:53:50,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:53:50,914 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:50,914 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-02 02:53:50,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:53:50,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:53:50,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-02 02:53:50,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:50,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:50,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:50,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:53:50,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-02 02:53:50,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:50,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:50,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:50,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,42019,1682996021676': 2023-05-02 02:53:50,918 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42019,1682996021676' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-05-02 02:53:50,918 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-05-02 02:53:50,918 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-02 02:53:50,918 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-02 02:53:50,918 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-05-02 02:53:50,918 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-02 02:53:50,920 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:50,920 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:50,920 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:50,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:50,921 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:50,921 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:50,920 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:50,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:50,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:50,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:53:50,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:53:50,921 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:50,921 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:50,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:50,922 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:50,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:53:50,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:50,922 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:50,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:50,922 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:50,922 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:50,922 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:50,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:50,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:50,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-02 02:53:50,922 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:50,923 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:50,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:50,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:53:50,923 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:50,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:50,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-02 02:53:50,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:50,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:50,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:50,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:53:50,925 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:50,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-02 02:53:50,925 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:50,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:50,926 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:50,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:50,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:50,928 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:50,928 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:50,928 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:50,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:50,929 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:50,928 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-02 02:53:50,929 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:50,928 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:50,929 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:50,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:50,928 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:50,929 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:50,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:50,929 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:50,929 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:50,929 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-05-02 02:53:50,929 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-02 02:53:50,929 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:50,929 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:50,929 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:50,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:50,930 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-02 02:53:50,929 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:50,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:50,930 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-05-02 02:53:50,930 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-02 02:53:50,930 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:50,930 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:50,930 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:50,930 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:50,930 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:50,930 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:50,930 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:50,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:51,030 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-02 02:53:51,032 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-02 02:53:51,050 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-02 02:53:51,052 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-05-02 02:53:51,052 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-05-02 02:53:51,053 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-05-02 02:53:51,053 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:51,054 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-05-02 02:53:51,054 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-05-02 02:53:51,054 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:51,054 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-02 02:53:51,055 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:51,055 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:51,055 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:51,055 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:51,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:51,055 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:51,055 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:51,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:51,056 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:51,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:51,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-02 02:53:51,056 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:51,056 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-02 02:53:51,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-02 02:53:51,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-02 02:53:51,056 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:51,056 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:51,057 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:51,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:53:51,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-02 02:53:51,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:53:51,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-02 02:53:51,057 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-02 02:53:51,057 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-02 02:53:51,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:53:51,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-02 02:53:51,057 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-02 02:53:51,057 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-05-02 02:53:51,057 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-05-02 02:53:51,057 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-05-02 02:53:51,057 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-02 02:53:51,057 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-02 02:53:51,057 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-02 02:53:51,057 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-02 02:53:51,057 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:51,057 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:51,058 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-02 02:53:51,059 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-02 02:53:51,059 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-02 02:53:51,057 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-02 02:53:51,059 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38925,1682996021633' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-02 02:53:51,059 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-02 02:53:51,059 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:51,059 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-02 02:53:51,061 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-02 02:53:51,061 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-02 02:53:51,061 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-02 02:53:51,061 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-02 02:53:51,061 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1682996029879.b92c1c7e8ba3ec18a0b4e94867da9855. 2023-05-02 02:53:51,062 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1682996029879.b92c1c7e8ba3ec18a0b4e94867da9855. started... 2023-05-02 02:53:51,062 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b92c1c7e8ba3ec18a0b4e94867da9855 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-02 02:53:51,066 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:51,066 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:51,066 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:51,066 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:53:51,066 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:51,067 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:53:51,073 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/.tmp/cf1/16a5e1fe22774dc782ac22bcb6a70060 2023-05-02 02:53:51,088 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/.tmp/cf2/488923b0b4ec41659120249ee07156c9 2023-05-02 02:53:51,094 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/.tmp/cf1/16a5e1fe22774dc782ac22bcb6a70060 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/16a5e1fe22774dc782ac22bcb6a70060 2023-05-02 02:53:51,100 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/16a5e1fe22774dc782ac22bcb6a70060, entries=10, sequenceid=45, filesize=5.0 K 2023-05-02 02:53:51,101 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/.tmp/cf2/488923b0b4ec41659120249ee07156c9 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/488923b0b4ec41659120249ee07156c9 2023-05-02 02:53:51,107 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/488923b0b4ec41659120249ee07156c9, entries=10, sequenceid=45, filesize=5.0 K 2023-05-02 02:53:51,109 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for b92c1c7e8ba3ec18a0b4e94867da9855 in 46ms, sequenceid=45, compaction requested=true 2023-05-02 02:53:51,109 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b92c1c7e8ba3ec18a0b4e94867da9855: 2023-05-02 02:53:51,109 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1682996029879.b92c1c7e8ba3ec18a0b4e94867da9855. 2023-05-02 02:53:51,109 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-02 02:53:51,109 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-02 02:53:51,109 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:51,109 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-02 02:53:51,109 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42019,1682996021676' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-02 02:53:51,110 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:51,110 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:51,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:51,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:53:51,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:53:51,111 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:51,111 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:53:51,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:53:51,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:53:51,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-02 02:53:51,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:51,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:51,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:51,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:53:51,113 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42019,1682996021676' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-05-02 02:53:51,113 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2b8af3e4[Count = 0] remaining members to acquire global barrier 2023-05-02 02:53:51,113 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-05-02 02:53:51,113 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:51,115 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:51,115 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:51,115 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:51,115 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:51,115 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:51,115 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:51,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:51,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:51,116 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-02 02:53:51,115 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:51,116 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-02 02:53:51,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:51,116 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-02 02:53:51,116 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-02 02:53:51,116 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-02 02:53:51,116 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-02 02:53:51,116 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-02 02:53:51,116 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,42019,1682996021676' in zk 2023-05-02 02:53:51,116 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,39081,1682996021579' in zk 2023-05-02 02:53:51,116 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,38925,1682996021633' in zk 2023-05-02 02:53:51,117 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:51,117 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-02 02:53:51,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:51,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:53:51,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:53:51,117 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:51,118 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-02 02:53:51,118 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-02 02:53:51,118 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-02 02:53:51,118 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:51,118 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:51,118 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-02 02:53:51,118 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-02 02:53:51,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:53:51,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:53:51,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-02 02:53:51,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:51,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:51,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:51,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:53:51,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-02 02:53:51,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:51,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:51,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:51,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,42019,1682996021676': 2023-05-02 02:53:51,121 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42019,1682996021676' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-05-02 02:53:51,121 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-05-02 02:53:51,121 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-02 02:53:51,121 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-02 02:53:51,121 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-05-02 02:53:51,121 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-02 02:53:51,123 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:51,123 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:51,123 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:51,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:51,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:53:51,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:53:51,123 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:51,123 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:51,123 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:51,123 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:51,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:51,123 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:51,123 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:51,123 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:51,123 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:51,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:51,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:53:51,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:51,124 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:51,124 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:51,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:51,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-02 02:53:51,124 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:51,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:51,124 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:51,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:51,124 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:51,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:53:51,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:51,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:51,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:51,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-02 02:53:51,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:51,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:51,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:51,126 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:51,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:53:51,126 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:51,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-02 02:53:51,127 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:51,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:51,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:51,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:51,131 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:51,131 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-02 02:53:51,131 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:51,131 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:51,131 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:51,131 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:51,131 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-05-02 02:53:51,131 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-02 02:53:51,131 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:51,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:51,131 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:51,131 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:51,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:51,131 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:51,131 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:51,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:51,131 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:51,131 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:51,132 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-05-02 02:53:51,132 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:51,132 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:51,132 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-02 02:53:51,132 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:51,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:51,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:51,132 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-02 02:53:51,132 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:51,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:51,132 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:51,133 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:51,133 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:51,133 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:51,133 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:51,133 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:51,233 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-02 02:53:51,234 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-02 02:53:51,253 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-02 02:53:51,255 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-05-02 02:53:51,256 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-05-02 02:53:51,256 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-05-02 02:53:51,256 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:51,256 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-05-02 02:53:51,256 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-05-02 02:53:51,256 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:51,256 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-02 02:53:51,257 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:51,257 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:51,257 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:51,258 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:51,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:51,257 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:51,257 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:51,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:51,258 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:51,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:51,258 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:51,258 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-02 02:53:51,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-02 02:53:51,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-02 02:53:51,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-02 02:53:51,258 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:51,259 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:51,259 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:51,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:53:51,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:53:51,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-02 02:53:51,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:53:51,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-02 02:53:51,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-02 02:53:51,259 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-02 02:53:51,259 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-02 02:53:51,259 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-02 02:53:51,259 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-05-02 02:53:51,259 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-02 02:53:51,259 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-05-02 02:53:51,260 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-02 02:53:51,260 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-02 02:53:51,260 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:51,260 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-02 02:53:51,260 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:51,260 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-02 02:53:51,260 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-02 02:53:51,260 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-05-02 02:53:51,260 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-02 02:53:51,260 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-02 02:53:51,260 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-02 02:53:51,260 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-02 02:53:51,260 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-02 02:53:51,260 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1682996029879.b92c1c7e8ba3ec18a0b4e94867da9855. 2023-05-02 02:53:51,260 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-02 02:53:51,260 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:51,260 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1682996029879.b92c1c7e8ba3ec18a0b4e94867da9855. started... 2023-05-02 02:53:51,260 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-02 02:53:51,261 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-02 02:53:51,261 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38925,1682996021633' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-02 02:53:51,261 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b92c1c7e8ba3ec18a0b4e94867da9855 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-02 02:53:51,261 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:51,262 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:51,262 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:51,263 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:53:51,263 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:51,263 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:53:51,281 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/.tmp/cf1/13ee2b4b0c024358a690edb0b94aac79 2023-05-02 02:53:51,296 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/.tmp/cf2/912fddaa1bc14af3ad9378909904f17f 2023-05-02 02:53:51,302 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/.tmp/cf1/13ee2b4b0c024358a690edb0b94aac79 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/13ee2b4b0c024358a690edb0b94aac79 2023-05-02 02:53:51,307 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/13ee2b4b0c024358a690edb0b94aac79, entries=10, sequenceid=58, filesize=5.0 K 2023-05-02 02:53:51,308 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/.tmp/cf2/912fddaa1bc14af3ad9378909904f17f as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/912fddaa1bc14af3ad9378909904f17f 2023-05-02 02:53:51,313 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/912fddaa1bc14af3ad9378909904f17f, entries=10, sequenceid=58, filesize=5.0 K 2023-05-02 02:53:51,315 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for b92c1c7e8ba3ec18a0b4e94867da9855 in 53ms, sequenceid=58, compaction requested=true 2023-05-02 02:53:51,315 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b92c1c7e8ba3ec18a0b4e94867da9855: 2023-05-02 02:53:51,315 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1682996029879.b92c1c7e8ba3ec18a0b4e94867da9855. 2023-05-02 02:53:51,315 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-02 02:53:51,315 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-02 02:53:51,315 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:51,315 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-02 02:53:51,315 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42019,1682996021676' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-02 02:53:51,318 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:51,318 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:51,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:51,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:53:51,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:53:51,318 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:51,318 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:53:51,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:53:51,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:53:51,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-02 02:53:51,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:51,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:51,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:51,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:53:51,320 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42019,1682996021676' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-05-02 02:53:51,320 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2ed2747a[Count = 0] remaining members to acquire global barrier 2023-05-02 02:53:51,320 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-05-02 02:53:51,321 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:51,322 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:51,322 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:51,322 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:51,322 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:51,322 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:51,322 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:51,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:51,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:51,322 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-02 02:53:51,322 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-02 02:53:51,322 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-02 02:53:51,322 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,42019,1682996021676' in zk 2023-05-02 02:53:51,322 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:51,322 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-02 02:53:51,322 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,38925,1682996021633' in zk 2023-05-02 02:53:51,322 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-02 02:53:51,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:51,322 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-02 02:53:51,323 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-02 02:53:51,323 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,39081,1682996021579' in zk 2023-05-02 02:53:51,324 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:51,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:51,324 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-02 02:53:51,324 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:51,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:53:51,324 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-02 02:53:51,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:53:51,324 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-02 02:53:51,324 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:51,324 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-02 02:53:51,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:53:51,326 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-02 02:53:51,326 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:51,326 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-02 02:53:51,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:53:51,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-02 02:53:51,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:51,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:51,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:51,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:53:51,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-02 02:53:51,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:51,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:51,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:51,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,42019,1682996021676': 2023-05-02 02:53:51,329 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42019,1682996021676' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-05-02 02:53:51,329 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-05-02 02:53:51,329 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-02 02:53:51,329 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-02 02:53:51,329 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-05-02 02:53:51,329 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-02 02:53:51,331 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:51,332 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:51,332 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:51,332 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:51,332 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:51,332 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:51,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:51,332 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:51,332 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:51,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:51,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:51,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:53:51,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:53:51,332 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:51,332 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:51,332 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:51,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:51,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:53:51,332 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:51,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:51,332 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:51,333 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:51,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-02 02:53:51,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:51,333 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:51,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:51,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:51,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:53:51,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:51,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:51,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-02 02:53:51,333 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:51,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:51,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:51,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:51,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:53:51,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-02 02:53:51,335 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:51,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:51,335 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:51,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:51,336 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:51,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:51,340 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:51,340 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:51,340 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:51,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:51,341 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:51,340 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:51,341 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:51,341 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:51,341 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:51,341 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-05-02 02:53:51,341 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-02 02:53:51,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:51,341 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:51,341 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-02 02:53:51,341 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:51,341 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:51,341 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:51,341 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:51,341 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:51,341 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-02 02:53:51,341 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-05-02 02:53:51,342 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-02 02:53:51,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:51,341 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:51,341 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:51,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:51,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:51,342 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:51,342 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:51,342 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:51,342 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:51,342 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:51,342 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:51,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:51,442 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-02 02:53:51,443 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-02 02:53:51,461 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-02 02:53:51,463 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-05-02 02:53:51,463 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-05-02 02:53:51,463 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-05-02 02:53:51,463 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:51,463 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-05-02 02:53:51,463 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-05-02 02:53:51,463 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:51,464 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-02 02:53:51,465 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:51,465 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:51,465 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:51,465 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:51,465 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:51,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:51,465 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:51,465 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-02 02:53:51,465 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:51,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:51,465 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:51,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:51,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-02 02:53:51,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-02 02:53:51,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-02 02:53:51,466 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:51,466 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:51,466 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:51,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:53:51,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-02 02:53:51,466 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-02 02:53:51,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:53:51,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-02 02:53:51,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:53:51,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-02 02:53:51,466 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-02 02:53:51,467 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-02 02:53:51,467 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-05-02 02:53:51,467 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-02 02:53:51,467 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-05-02 02:53:51,467 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-02 02:53:51,467 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:51,467 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-05-02 02:53:51,467 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-02 02:53:51,467 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-02 02:53:51,467 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38925,1682996021633' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-02 02:53:51,467 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-02 02:53:51,467 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-02 02:53:51,467 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-02 02:53:51,467 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-02 02:53:51,467 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:51,467 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:51,467 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-02 02:53:51,467 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-02 02:53:51,467 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-02 02:53:51,467 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-02 02:53:51,467 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-02 02:53:51,468 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1682996029879.b92c1c7e8ba3ec18a0b4e94867da9855. 2023-05-02 02:53:51,468 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-02 02:53:51,468 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1682996029879.b92c1c7e8ba3ec18a0b4e94867da9855. started... 2023-05-02 02:53:51,468 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b92c1c7e8ba3ec18a0b4e94867da9855 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-02 02:53:51,469 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:51,470 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:51,470 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:51,470 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:53:51,471 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:51,471 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:53:51,479 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/.tmp/cf1/4d8bff8f40d4435e82f15595c5237edd 2023-05-02 02:53:51,494 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/.tmp/cf2/53ab7c6e85de44f29fcbaf368e39d276 2023-05-02 02:53:51,499 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/.tmp/cf1/4d8bff8f40d4435e82f15595c5237edd as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/4d8bff8f40d4435e82f15595c5237edd 2023-05-02 02:53:51,510 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/4d8bff8f40d4435e82f15595c5237edd, entries=10, sequenceid=71, filesize=5.0 K 2023-05-02 02:53:51,512 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/.tmp/cf2/53ab7c6e85de44f29fcbaf368e39d276 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/53ab7c6e85de44f29fcbaf368e39d276 2023-05-02 02:53:51,518 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/53ab7c6e85de44f29fcbaf368e39d276, entries=10, sequenceid=71, filesize=5.0 K 2023-05-02 02:53:51,519 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for b92c1c7e8ba3ec18a0b4e94867da9855 in 51ms, sequenceid=71, compaction requested=true 2023-05-02 02:53:51,519 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b92c1c7e8ba3ec18a0b4e94867da9855: 2023-05-02 02:53:51,519 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1682996029879.b92c1c7e8ba3ec18a0b4e94867da9855. 2023-05-02 02:53:51,519 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-02 02:53:51,519 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-02 02:53:51,519 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:51,519 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-02 02:53:51,519 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42019,1682996021676' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-02 02:53:51,521 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:51,521 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:51,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:51,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:53:51,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:53:51,521 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:51,521 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:53:51,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:53:51,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:53:51,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-02 02:53:51,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:51,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:51,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:51,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:53:51,524 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42019,1682996021676' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-05-02 02:53:51,524 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6df61ebd[Count = 0] remaining members to acquire global barrier 2023-05-02 02:53:51,524 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-05-02 02:53:51,524 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:51,525 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:51,525 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:51,525 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:51,525 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:51,525 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:51,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:51,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:51,526 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-02 02:53:51,526 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-02 02:53:51,526 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-02 02:53:51,526 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:51,526 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:51,526 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-02 02:53:51,526 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,42019,1682996021676' in zk 2023-05-02 02:53:51,526 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-02 02:53:51,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:51,526 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,38925,1682996021633' in zk 2023-05-02 02:53:51,526 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-02 02:53:51,526 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-02 02:53:51,526 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,39081,1682996021579' in zk 2023-05-02 02:53:51,528 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-02 02:53:51,528 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-02 02:53:51,528 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:51,528 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-02 02:53:51,528 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:51,528 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:51,528 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-02 02:53:51,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:51,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:53:51,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:53:51,528 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:51,528 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-02 02:53:51,528 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-02 02:53:51,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:53:51,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:53:51,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-02 02:53:51,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:51,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:51,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:51,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:53:51,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-02 02:53:51,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:51,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:51,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:51,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,42019,1682996021676': 2023-05-02 02:53:51,532 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42019,1682996021676' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-05-02 02:53:51,532 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-05-02 02:53:51,532 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-02 02:53:51,532 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-02 02:53:51,532 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-05-02 02:53:51,532 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-02 02:53:51,535 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:51,535 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:51,535 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:51,535 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:51,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:51,535 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:51,535 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:51,535 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:51,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:51,535 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:51,535 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:51,536 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:51,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:51,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:53:51,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:53:51,535 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:51,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:51,536 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:51,536 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:51,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:51,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:51,536 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:51,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:51,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:53:51,537 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:51,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:51,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:51,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:51,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-02 02:53:51,537 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:51,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:53:51,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-02 02:53:51,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:51,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:51,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:51,539 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:51,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:53:51,539 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:51,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-02 02:53:51,540 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:51,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:51,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:51,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:51,543 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:51,543 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:51,543 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:51,543 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-02 02:53:51,543 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:51,543 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:51,543 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:51,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:51,543 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:51,543 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:51,543 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:51,544 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:51,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:51,544 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-05-02 02:53:51,544 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-02 02:53:51,544 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:51,544 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:51,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:51,544 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:51,544 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:51,544 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:51,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:51,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:51,544 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-02 02:53:51,544 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-05-02 02:53:51,544 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:51,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:51,544 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:51,545 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-02 02:53:51,545 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:51,545 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:51,545 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:51,545 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-02 02:53:51,545 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-02 02:53:51,645 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-02 02:53:51,646 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-02 02:53:51,908 WARN [Listener at localhost/37119] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-05-02 02:53:51,910 DEBUG [Listener at localhost/37119] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:51,911 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(1029): Stored pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:51,912 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:51,912 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:51,912 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:51,914 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-02 02:53:51,914 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:51,914 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-02 02:53:51,914 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-02 02:53:52,162 INFO [Listener at localhost/37119] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:52,163 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-02 02:53:52,163 INFO [Listener at localhost/37119] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-02 02:53:52,163 INFO [Listener at localhost/37119] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-02 02:53:52,165 ERROR [Listener at localhost/37119] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-1,5,PEWorkerGroup] Thread[HFileArchiver-2,5,PEWorkerGroup] Thread[HFileArchiver-3,5,PEWorkerGroup] Thread[HFileArchiver-4,5,PEWorkerGroup] Thread[HFileArchiver-5,5,PEWorkerGroup] Thread[HFileArchiver-6,5,PEWorkerGroup] 2023-05-02 02:53:52,166 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-02 02:53:52,166 INFO [Listener at localhost/37119] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-02 02:53:52,166 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-02 02:53:52,166 INFO [Listener at localhost/37119] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-02 02:53:52,167 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-02 02:53:52,171 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-05-02 02:53:52,173 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-02 02:53:52,173 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-02 02:53:52,174 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-02 02:53:52,174 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-02 02:53:52,175 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:53:52,175 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=084b1607f7ed8615120a04e4729fbafa, daughterA=ea7683cd3436a62eb16418a3c875d563, daughterB=408e891cab9b919f4038b233ba34779d 2023-05-02 02:53:52,176 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:53:52,176 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:53:52,177 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-02 02:53:52,177 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:52,178 DEBUG [Listener at localhost/37119] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e held the lock before restarting, call acquireLock to restore it. 2023-05-02 02:53:52,178 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:52,178 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:52,178 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:52,179 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 11 msec 2023-05-02 02:53:52,179 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-02 02:53:52,180 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,38925,1682996021633, table=hbase:meta, region=1588230740 2023-05-02 02:53:52,182 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682996062181 2023-05-02 02:53:52,183 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:52,183 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(1517): Joining cluster... 2023-05-02 02:53:52,187 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=5c823d6f15ad7b8fe82f9ae4beee6513, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42019,1682996021676, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676, openSeqNum=2 2023-05-02 02:53:52,188 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=b92c1c7e8ba3ec18a0b4e94867da9855, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42019,1682996021676, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676, openSeqNum=2 2023-05-02 02:53:52,188 INFO [Listener at localhost/37119] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-02 02:53:52,188 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682996092188 2023-05-02 02:53:52,188 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682996152188 2023-05-02 02:53:52,188 INFO [Listener at localhost/37119] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-05-02 02:53:52,188 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-02 02:53:52,191 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-02 02:53:52,191 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:52,191 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-02 02:53:52,191 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-02 02:53:52,438 INFO [Listener at localhost/37119] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:52,439 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-02 02:53:52,439 INFO [Listener at localhost/37119] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-02 02:53:52,439 INFO [Listener at localhost/37119] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-02 02:53:52,439 ERROR [Listener at localhost/37119] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-02 02:53:52,439 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-02 02:53:52,439 INFO [Listener at localhost/37119] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-02 02:53:52,439 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-02 02:53:52,439 INFO [Listener at localhost/37119] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-02 02:53:52,440 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-02 02:53:52,442 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-05-02 02:53:52,443 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-02 02:53:52,444 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-02 02:53:52,444 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-02 02:53:52,444 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-02 02:53:52,445 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:53:52,445 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=084b1607f7ed8615120a04e4729fbafa, daughterA=ea7683cd3436a62eb16418a3c875d563, daughterB=408e891cab9b919f4038b233ba34779d 2023-05-02 02:53:52,446 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:53:52,446 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:53:52,446 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-02 02:53:52,447 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:52,447 DEBUG [Listener at localhost/37119] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e held the lock before restarting, call acquireLock to restore it. 2023-05-02 02:53:52,447 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:52,447 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:52,447 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:52,447 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-05-02 02:53:52,448 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-02 02:53:52,449 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,38925,1682996021633, table=hbase:meta, region=1588230740 2023-05-02 02:53:52,449 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682996062449 2023-05-02 02:53:52,451 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:52,451 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(1517): Joining cluster... 2023-05-02 02:53:52,454 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=5c823d6f15ad7b8fe82f9ae4beee6513, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42019,1682996021676, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676, openSeqNum=2 2023-05-02 02:53:52,454 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=b92c1c7e8ba3ec18a0b4e94867da9855, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42019,1682996021676, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676, openSeqNum=2 2023-05-02 02:53:52,454 INFO [Listener at localhost/37119] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-02 02:53:52,454 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682996092454 2023-05-02 02:53:52,454 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682996152454 2023-05-02 02:53:52,454 INFO [Listener at localhost/37119] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-02 02:53:52,455 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-02 02:53:52,457 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=b92c1c7e8ba3ec18a0b4e94867da9855, UNASSIGN}] 2023-05-02 02:53:52,457 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-02 02:53:52,457 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:52,457 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-02 02:53:52,457 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-02 02:53:52,705 INFO [Listener at localhost/37119] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:52,705 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-02 02:53:52,705 INFO [Listener at localhost/37119] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-02 02:53:52,705 INFO [Listener at localhost/37119] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-02 02:53:52,705 ERROR [Listener at localhost/37119] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-02 02:53:52,705 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-02 02:53:52,705 INFO [Listener at localhost/37119] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-02 02:53:52,705 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-02 02:53:52,705 INFO [Listener at localhost/37119] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-02 02:53:52,706 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-02 02:53:52,708 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-05-02 02:53:52,708 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-02 02:53:52,709 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-02 02:53:52,709 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-02 02:53:52,709 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-02 02:53:52,710 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:53:52,710 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=084b1607f7ed8615120a04e4729fbafa, daughterA=ea7683cd3436a62eb16418a3c875d563, daughterB=408e891cab9b919f4038b233ba34779d 2023-05-02 02:53:52,711 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:53:52,711 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:53:52,711 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-02 02:53:52,712 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:52,712 DEBUG [Listener at localhost/37119] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e held the lock before restarting, call acquireLock to restore it. 2023-05-02 02:53:52,712 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:52,712 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:52,712 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:52,712 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-05-02 02:53:52,713 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-02 02:53:52,713 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,38925,1682996021633, table=hbase:meta, region=1588230740 2023-05-02 02:53:52,713 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682996062713 2023-05-02 02:53:52,715 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(1517): Joining cluster... 2023-05-02 02:53:52,715 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:52,718 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=5c823d6f15ad7b8fe82f9ae4beee6513, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42019,1682996021676, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676, openSeqNum=2 2023-05-02 02:53:52,718 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=b92c1c7e8ba3ec18a0b4e94867da9855, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42019,1682996021676, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676, openSeqNum=2 2023-05-02 02:53:52,718 INFO [Listener at localhost/37119] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-02 02:53:52,718 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682996092718 2023-05-02 02:53:52,718 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682996152718 2023-05-02 02:53:52,718 INFO [Listener at localhost/37119] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-02 02:53:52,718 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=b92c1c7e8ba3ec18a0b4e94867da9855, UNASSIGN}] 2023-05-02 02:53:52,718 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-02 02:53:52,721 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=b92c1c7e8ba3ec18a0b4e94867da9855, UNASSIGN 2023-05-02 02:53:52,722 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=b92c1c7e8ba3ec18a0b4e94867da9855, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:52,722 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1682996029879.b92c1c7e8ba3ec18a0b4e94867da9855.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1682996032722"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1682996032722"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682996032722"}]},"ts":"1682996032722"} 2023-05-02 02:53:52,723 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure b92c1c7e8ba3ec18a0b4e94867da9855, server=jenkins-hbase4.apache.org,42019,1682996021676}] 2023-05-02 02:53:52,723 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-02 02:53:52,723 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=b92c1c7e8ba3ec18a0b4e94867da9855, UNASSIGN 2023-05-02 02:53:52,723 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-02 02:53:52,724 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=b92c1c7e8ba3ec18a0b4e94867da9855, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-02 02:53:52,968 INFO [Listener at localhost/37119] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:52,968 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-02 02:53:52,969 INFO [Listener at localhost/37119] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-02 02:53:52,969 INFO [Listener at localhost/37119] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-02 02:53:52,969 ERROR [Listener at localhost/37119] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-02 02:53:52,969 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-02 02:53:52,969 INFO [Listener at localhost/37119] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-02 02:53:52,969 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-02 02:53:52,969 INFO [Listener at localhost/37119] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-02 02:53:52,970 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-02 02:53:52,971 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-05-02 02:53:52,972 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-05-02 02:53:52,972 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-02 02:53:52,972 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-02 02:53:52,972 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-02 02:53:52,973 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-02 02:53:52,973 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:53:52,974 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=084b1607f7ed8615120a04e4729fbafa, daughterA=ea7683cd3436a62eb16418a3c875d563, daughterB=408e891cab9b919f4038b233ba34779d 2023-05-02 02:53:52,974 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:53:52,974 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:53:52,974 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-02 02:53:52,975 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:52,975 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=b92c1c7e8ba3ec18a0b4e94867da9855, UNASSIGN 2023-05-02 02:53:52,975 DEBUG [Listener at localhost/37119] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e held the lock before restarting, call acquireLock to restore it. 2023-05-02 02:53:52,975 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:52,976 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:52,976 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:52,976 DEBUG [Listener at localhost/37119] procedure2.Procedure(966): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=b92c1c7e8ba3ec18a0b4e94867da9855, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-05-02 02:53:52,976 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=b92c1c7e8ba3ec18a0b4e94867da9855, UNASSIGN 2023-05-02 02:53:52,976 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-05-02 02:53:52,976 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-02 02:53:52,977 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,38925,1682996021633, table=hbase:meta, region=1588230740 2023-05-02 02:53:52,977 INFO [Listener at localhost/37119] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=b92c1c7e8ba3ec18a0b4e94867da9855, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=b92c1c7e8ba3ec18a0b4e94867da9855 to restore RIT 2023-05-02 02:53:52,977 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682996062977 2023-05-02 02:53:52,978 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(1517): Joining cluster... 2023-05-02 02:53:52,978 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=b92c1c7e8ba3ec18a0b4e94867da9855, UNASSIGN 2023-05-02 02:53:52,981 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=5c823d6f15ad7b8fe82f9ae4beee6513, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42019,1682996021676, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676, openSeqNum=2 2023-05-02 02:53:52,981 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=b92c1c7e8ba3ec18a0b4e94867da9855, regionState=CLOSING, lastHost=jenkins-hbase4.apache.org,42019,1682996021676, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676, openSeqNum=2 2023-05-02 02:53:52,981 INFO [Listener at localhost/37119] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-02 02:53:52,981 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682996092981 2023-05-02 02:53:52,981 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682996152981 2023-05-02 02:53:52,981 INFO [Listener at localhost/37119] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-02 02:53:52,982 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=b92c1c7e8ba3ec18a0b4e94867da9855, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:52,982 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1682996029879.b92c1c7e8ba3ec18a0b4e94867da9855.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1682996032982"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1682996032982"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682996032982"}]},"ts":"1682996032982"} 2023-05-02 02:53:52,983 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure b92c1c7e8ba3ec18a0b4e94867da9855, server=jenkins-hbase4.apache.org,42019,1682996021676}] 2023-05-02 02:53:52,983 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-02 02:53:53,137 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:53,137 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing b92c1c7e8ba3ec18a0b4e94867da9855, disabling compactions & flushes 2023-05-02 02:53:53,137 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitWithoutPONR,,1682996029879.b92c1c7e8ba3ec18a0b4e94867da9855. 2023-05-02 02:53:53,137 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitWithoutPONR,,1682996029879.b92c1c7e8ba3ec18a0b4e94867da9855. 2023-05-02 02:53:53,137 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitWithoutPONR,,1682996029879.b92c1c7e8ba3ec18a0b4e94867da9855. after waiting 0 ms 2023-05-02 02:53:53,137 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitWithoutPONR,,1682996029879.b92c1c7e8ba3ec18a0b4e94867da9855. 2023-05-02 02:53:53,137 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing b92c1c7e8ba3ec18a0b4e94867da9855 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-05-02 02:53:53,148 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/.tmp/cf1/8dd2e18ff3e64f1999a3bee1527292e3 2023-05-02 02:53:53,166 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/.tmp/cf2/427ba387297e4b318fda28a69d090362 2023-05-02 02:53:53,172 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/.tmp/cf1/8dd2e18ff3e64f1999a3bee1527292e3 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/8dd2e18ff3e64f1999a3bee1527292e3 2023-05-02 02:53:53,177 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/8dd2e18ff3e64f1999a3bee1527292e3, entries=8, sequenceid=82, filesize=5.0 K 2023-05-02 02:53:53,178 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/.tmp/cf2/427ba387297e4b318fda28a69d090362 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/427ba387297e4b318fda28a69d090362 2023-05-02 02:53:53,184 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/427ba387297e4b318fda28a69d090362, entries=8, sequenceid=82, filesize=5.0 K 2023-05-02 02:53:53,185 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for b92c1c7e8ba3ec18a0b4e94867da9855 in 48ms, sequenceid=82, compaction requested=true 2023-05-02 02:53:53,208 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-05-02 02:53:53,209 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-02 02:53:53,209 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitWithoutPONR,,1682996029879.b92c1c7e8ba3ec18a0b4e94867da9855. 2023-05-02 02:53:53,209 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for b92c1c7e8ba3ec18a0b4e94867da9855: 2023-05-02 02:53:53,211 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:53,212 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=b92c1c7e8ba3ec18a0b4e94867da9855, regionState=CLOSED 2023-05-02 02:53:53,212 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1682996029879.b92c1c7e8ba3ec18a0b4e94867da9855.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1682996033212"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682996033212"}]},"ts":"1682996033212"} 2023-05-02 02:53:53,214 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-02 02:53:53,214 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure b92c1c7e8ba3ec18a0b4e94867da9855, server=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:53,214 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-02 02:53:53,214 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure b92c1c7e8ba3ec18a0b4e94867da9855, server=jenkins-hbase4.apache.org,42019,1682996021676 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-02 02:53:53,232 INFO [Listener at localhost/37119] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:53,232 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-02 02:53:53,232 INFO [Listener at localhost/37119] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-02 02:53:53,232 INFO [Listener at localhost/37119] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-02 02:53:53,232 ERROR [Listener at localhost/37119] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-02 02:53:53,233 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-02 02:53:53,233 INFO [Listener at localhost/37119] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-02 02:53:53,233 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-02 02:53:53,233 INFO [Listener at localhost/37119] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-02 02:53:53,234 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-02 02:53:53,235 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-05-02 02:53:53,235 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-05-02 02:53:53,235 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-05-02 02:53:53,236 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-02 02:53:53,236 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-02 02:53:53,236 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-02 02:53:53,236 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-02 02:53:53,237 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:53:53,237 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=084b1607f7ed8615120a04e4729fbafa, daughterA=ea7683cd3436a62eb16418a3c875d563, daughterB=408e891cab9b919f4038b233ba34779d 2023-05-02 02:53:53,238 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:53:53,238 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:53:53,238 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-02 02:53:53,239 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:53,239 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=b92c1c7e8ba3ec18a0b4e94867da9855, UNASSIGN 2023-05-02 02:53:53,239 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure b92c1c7e8ba3ec18a0b4e94867da9855, server=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:53,239 DEBUG [Listener at localhost/37119] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e held the lock before restarting, call acquireLock to restore it. 2023-05-02 02:53:53,239 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:53,239 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:53,239 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:53,239 DEBUG [Listener at localhost/37119] procedure2.Procedure(966): pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=b92c1c7e8ba3ec18a0b4e94867da9855, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-05-02 02:53:53,239 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=b92c1c7e8ba3ec18a0b4e94867da9855, UNASSIGN 2023-05-02 02:53:53,240 DEBUG [Listener at localhost/37119] procedure2.Procedure(966): pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure b92c1c7e8ba3ec18a0b4e94867da9855, server=jenkins-hbase4.apache.org,42019,1682996021676 held the lock before restarting, call acquireLock to restore it. 2023-05-02 02:53:53,240 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-05-02 02:53:53,240 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-02 02:53:53,241 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,38925,1682996021633, table=hbase:meta, region=1588230740 2023-05-02 02:53:53,241 INFO [Listener at localhost/37119] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=b92c1c7e8ba3ec18a0b4e94867da9855, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=b92c1c7e8ba3ec18a0b4e94867da9855 to restore RIT 2023-05-02 02:53:53,243 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682996063243 2023-05-02 02:53:53,243 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(1517): Joining cluster... 2023-05-02 02:53:53,244 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure b92c1c7e8ba3ec18a0b4e94867da9855, server=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:53,247 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=5c823d6f15ad7b8fe82f9ae4beee6513, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42019,1682996021676, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676, openSeqNum=2 2023-05-02 02:53:53,247 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=b92c1c7e8ba3ec18a0b4e94867da9855, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,42019,1682996021676, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676, openSeqNum=2 2023-05-02 02:53:53,247 INFO [Listener at localhost/37119] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-02 02:53:53,247 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682996093247 2023-05-02 02:53:53,247 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682996153247 2023-05-02 02:53:53,247 INFO [Listener at localhost/37119] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-02 02:53:53,247 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=b92c1c7e8ba3ec18a0b4e94867da9855, regionState=CLOSED, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:53,248 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1682996029879.b92c1c7e8ba3ec18a0b4e94867da9855.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1682996033247"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1682996033247"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682996033247"}]},"ts":"1682996033247"} 2023-05-02 02:53:53,249 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-02 02:53:53,252 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=30, resume processing ppid=29 2023-05-02 02:53:53,252 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure b92c1c7e8ba3ec18a0b4e94867da9855, server=jenkins-hbase4.apache.org,42019,1682996021676 in 266 msec 2023-05-02 02:53:53,252 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-02 02:53:53,252 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=b92c1c7e8ba3ec18a0b4e94867da9855, UNASSIGN 2023-05-02 02:53:53,253 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-02 02:53:53,253 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=b92c1c7e8ba3ec18a0b4e94867da9855, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-02 02:53:53,497 INFO [Listener at localhost/37119] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:53,498 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-02 02:53:53,498 INFO [Listener at localhost/37119] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-02 02:53:53,498 INFO [Listener at localhost/37119] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-02 02:53:53,498 ERROR [Listener at localhost/37119] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-02 02:53:53,498 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-02 02:53:53,498 INFO [Listener at localhost/37119] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-02 02:53:53,498 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-02 02:53:53,498 INFO [Listener at localhost/37119] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-02 02:53:53,499 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-02 02:53:53,500 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-05-02 02:53:53,500 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-05-02 02:53:53,500 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-02 02:53:53,501 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-02 02:53:53,501 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-02 02:53:53,502 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-02 02:53:53,502 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-02 02:53:53,502 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:53:53,502 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=084b1607f7ed8615120a04e4729fbafa, daughterA=ea7683cd3436a62eb16418a3c875d563, daughterB=408e891cab9b919f4038b233ba34779d 2023-05-02 02:53:53,503 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:53:53,503 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:53:53,503 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-02 02:53:53,504 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:53,504 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=b92c1c7e8ba3ec18a0b4e94867da9855, UNASSIGN 2023-05-02 02:53:53,504 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure b92c1c7e8ba3ec18a0b4e94867da9855, server=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:53,504 DEBUG [Listener at localhost/37119] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e held the lock before restarting, call acquireLock to restore it. 2023-05-02 02:53:53,504 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:53,504 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:53,504 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:53,504 DEBUG [Listener at localhost/37119] procedure2.Procedure(946): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=b92c1c7e8ba3ec18a0b4e94867da9855, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-02 02:53:53,504 DEBUG [Listener at localhost/37119] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure b92c1c7e8ba3ec18a0b4e94867da9855, server=jenkins-hbase4.apache.org,42019,1682996021676 didn't hold the lock before restarting, skip acquiring lock. 2023-05-02 02:53:53,505 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-05-02 02:53:53,505 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-02 02:53:53,506 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,38925,1682996021633, table=hbase:meta, region=1588230740 2023-05-02 02:53:53,506 INFO [Listener at localhost/37119] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=b92c1c7e8ba3ec18a0b4e94867da9855, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=b92c1c7e8ba3ec18a0b4e94867da9855 to restore RIT 2023-05-02 02:53:53,506 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682996063506 2023-05-02 02:53:53,507 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=b92c1c7e8ba3ec18a0b4e94867da9855, UNASSIGN 2023-05-02 02:53:53,507 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(1517): Joining cluster... 2023-05-02 02:53:53,510 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=5c823d6f15ad7b8fe82f9ae4beee6513, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42019,1682996021676, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676, openSeqNum=2 2023-05-02 02:53:53,510 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=b92c1c7e8ba3ec18a0b4e94867da9855, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,42019,1682996021676, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676, openSeqNum=2 2023-05-02 02:53:53,510 INFO [Listener at localhost/37119] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-02 02:53:53,510 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682996093510 2023-05-02 02:53:53,510 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682996153510 2023-05-02 02:53:53,510 INFO [Listener at localhost/37119] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-02 02:53:53,510 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=b92c1c7e8ba3ec18a0b4e94867da9855, UNASSIGN 2023-05-02 02:53:53,512 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-02 02:53:53,516 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=29, resume processing ppid=28 2023-05-02 02:53:53,516 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=b92c1c7e8ba3ec18a0b4e94867da9855, UNASSIGN in 794 msec 2023-05-02 02:53:53,519 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-02 02:53:53,519 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:53,519 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-02 02:53:53,519 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-02 02:53:53,761 INFO [Listener at localhost/37119] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:53,761 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-02 02:53:53,761 INFO [Listener at localhost/37119] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-02 02:53:53,761 INFO [Listener at localhost/37119] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-02 02:53:53,761 ERROR [Listener at localhost/37119] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-02 02:53:53,761 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-02 02:53:53,761 INFO [Listener at localhost/37119] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-02 02:53:53,761 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-02 02:53:53,761 INFO [Listener at localhost/37119] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-02 02:53:53,762 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-02 02:53:53,763 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-05-02 02:53:53,763 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-02 02:53:53,764 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-02 02:53:53,764 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-02 02:53:53,764 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-02 02:53:53,764 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-02 02:53:53,765 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-02 02:53:53,765 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:53:53,765 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=084b1607f7ed8615120a04e4729fbafa, daughterA=ea7683cd3436a62eb16418a3c875d563, daughterB=408e891cab9b919f4038b233ba34779d 2023-05-02 02:53:53,765 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:53:53,766 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:53:53,766 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-02 02:53:53,766 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:53,767 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=b92c1c7e8ba3ec18a0b4e94867da9855, UNASSIGN 2023-05-02 02:53:53,767 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure b92c1c7e8ba3ec18a0b4e94867da9855, server=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:53,767 DEBUG [Listener at localhost/37119] procedure2.Procedure(946): pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e didn't hold the lock before restarting, skip acquiring lock. 2023-05-02 02:53:53,767 DEBUG [Listener at localhost/37119] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=b92c1c7e8ba3ec18a0b4e94867da9855, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-02 02:53:53,767 DEBUG [Listener at localhost/37119] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure b92c1c7e8ba3ec18a0b4e94867da9855, server=jenkins-hbase4.apache.org,42019,1682996021676 didn't hold the lock before restarting, skip acquiring lock. 2023-05-02 02:53:53,767 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-05-02 02:53:53,768 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-02 02:53:53,768 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,38925,1682996021633, table=hbase:meta, region=1588230740 2023-05-02 02:53:53,768 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682996063768 2023-05-02 02:53:53,770 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(1517): Joining cluster... 2023-05-02 02:53:53,770 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:53,772 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=5c823d6f15ad7b8fe82f9ae4beee6513, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42019,1682996021676, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676, openSeqNum=2 2023-05-02 02:53:53,773 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=b92c1c7e8ba3ec18a0b4e94867da9855, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,42019,1682996021676, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676, openSeqNum=2 2023-05-02 02:53:53,773 INFO [Listener at localhost/37119] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-02 02:53:53,773 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682996093773 2023-05-02 02:53:53,773 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682996153773 2023-05-02 02:53:53,773 INFO [Listener at localhost/37119] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-05-02 02:53:53,773 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:53,773 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:53,773 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:53,778 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-02 02:53:53,787 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=b92c1c7e8ba3ec18a0b4e94867da9855, threads=14 2023-05-02 02:53:53,788 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/427ba387297e4b318fda28a69d090362 for region: b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:53,789 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/488923b0b4ec41659120249ee07156c9 for region: b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:53,789 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/53ab7c6e85de44f29fcbaf368e39d276 for region: b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:53,789 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/5c6252080bbc48e6b6a023e08c36b716 for region: b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:53,789 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/8f0768252daf4fca8efafdcc6273b3c6 for region: b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:53,789 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/912fddaa1bc14af3ad9378909904f17f for region: b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:53,789 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/13ee2b4b0c024358a690edb0b94aac79 for region: b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:53,789 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/b94fd300967f43f7a6ca454238517875 for region: b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:53,790 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/16a5e1fe22774dc782ac22bcb6a70060 for region: b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:53,790 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/4d8bff8f40d4435e82f15595c5237edd for region: b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:53,790 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/8dd2e18ff3e64f1999a3bee1527292e3 for region: b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:53,790 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/a78695d99c44472fb73a8df5d283aed5 for region: b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:53,791 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/5317f0b5b225487c82a9f79b071b429d for region: b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:53,791 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/de0cd278f5ae475c8e64f02e18179538 for region: b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:53,987 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/8f0768252daf4fca8efafdcc6273b3c6 for region: b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:53,987 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/427ba387297e4b318fda28a69d090362 for region: b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:53,989 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/53ab7c6e85de44f29fcbaf368e39d276 for region: b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:53,992 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/a78695d99c44472fb73a8df5d283aed5 for region: b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:54,002 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/912fddaa1bc14af3ad9378909904f17f for region: b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:54,011 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/5c6252080bbc48e6b6a023e08c36b716 for region: b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:54,016 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/5317f0b5b225487c82a9f79b071b429d for region: b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:54,016 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/13ee2b4b0c024358a690edb0b94aac79 for region: b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:54,016 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/16a5e1fe22774dc782ac22bcb6a70060 for region: b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:54,031 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/b94fd300967f43f7a6ca454238517875 for region: b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:54,037 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/4d8bff8f40d4435e82f15595c5237edd for region: b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:54,037 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/8dd2e18ff3e64f1999a3bee1527292e3 for region: b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:54,042 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/de0cd278f5ae475c8e64f02e18179538 for region: b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:54,130 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-05-02 02:53:54,403 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/488923b0b4ec41659120249ee07156c9 for region: b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:54,403 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region b92c1c7e8ba3ec18a0b4e94867da9855 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-05-02 02:53:54,449 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-02 02:53:54,449 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:54,449 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-02 02:53:54,449 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-02 02:53:54,523 INFO [Listener at localhost/37119] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:54,523 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-02 02:53:54,523 INFO [Listener at localhost/37119] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-02 02:53:54,524 INFO [Listener at localhost/37119] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-02 02:53:54,524 ERROR [Listener at localhost/37119] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-02 02:53:54,524 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-02 02:53:54,524 INFO [Listener at localhost/37119] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-02 02:53:54,524 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-02 02:53:54,524 INFO [Listener at localhost/37119] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-02 02:53:54,525 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-02 02:53:54,526 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-05-02 02:53:54,526 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-02 02:53:54,526 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-02 02:53:54,527 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-02 02:53:54,527 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-02 02:53:54,527 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-02 02:53:54,528 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-02 02:53:54,528 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:53:54,529 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=084b1607f7ed8615120a04e4729fbafa, daughterA=ea7683cd3436a62eb16418a3c875d563, daughterB=408e891cab9b919f4038b233ba34779d 2023-05-02 02:53:54,529 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:53:54,529 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:53:54,530 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-02 02:53:54,530 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:54,530 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=b92c1c7e8ba3ec18a0b4e94867da9855, UNASSIGN 2023-05-02 02:53:54,531 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure b92c1c7e8ba3ec18a0b4e94867da9855, server=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:54,531 DEBUG [Listener at localhost/37119] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e held the lock before restarting, call acquireLock to restore it. 2023-05-02 02:53:54,531 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:54,531 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:54,531 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:54,531 DEBUG [Listener at localhost/37119] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=b92c1c7e8ba3ec18a0b4e94867da9855, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-02 02:53:54,531 DEBUG [Listener at localhost/37119] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure b92c1c7e8ba3ec18a0b4e94867da9855, server=jenkins-hbase4.apache.org,42019,1682996021676 didn't hold the lock before restarting, skip acquiring lock. 2023-05-02 02:53:54,531 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-05-02 02:53:54,532 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-02 02:53:54,532 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,38925,1682996021633, table=hbase:meta, region=1588230740 2023-05-02 02:53:54,532 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682996064532 2023-05-02 02:53:54,534 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(1517): Joining cluster... 2023-05-02 02:53:54,534 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:54,536 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=5c823d6f15ad7b8fe82f9ae4beee6513, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42019,1682996021676, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676, openSeqNum=2 2023-05-02 02:53:54,536 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=b92c1c7e8ba3ec18a0b4e94867da9855, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,42019,1682996021676, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676, openSeqNum=2 2023-05-02 02:53:54,537 INFO [Listener at localhost/37119] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-02 02:53:54,537 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682996094537 2023-05-02 02:53:54,537 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682996154537 2023-05-02 02:53:54,537 INFO [Listener at localhost/37119] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-05-02 02:53:54,547 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=b92c1c7e8ba3ec18a0b4e94867da9855, threads=14 2023-05-02 02:53:54,548 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/427ba387297e4b318fda28a69d090362 for region: b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:54,549 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/488923b0b4ec41659120249ee07156c9 for region: b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:54,549 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/53ab7c6e85de44f29fcbaf368e39d276 for region: b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:54,549 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/5c6252080bbc48e6b6a023e08c36b716 for region: b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:54,550 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/8f0768252daf4fca8efafdcc6273b3c6 for region: b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:54,550 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/912fddaa1bc14af3ad9378909904f17f for region: b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:54,550 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/b94fd300967f43f7a6ca454238517875 for region: b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:54,551 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/13ee2b4b0c024358a690edb0b94aac79 for region: b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:54,551 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/16a5e1fe22774dc782ac22bcb6a70060 for region: b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:54,551 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/4d8bff8f40d4435e82f15595c5237edd for region: b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:54,552 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/5317f0b5b225487c82a9f79b071b429d for region: b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:54,553 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/8dd2e18ff3e64f1999a3bee1527292e3 for region: b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:54,555 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/a78695d99c44472fb73a8df5d283aed5 for region: b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:54,556 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/de0cd278f5ae475c8e64f02e18179538 for region: b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:54,745 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/427ba387297e4b318fda28a69d090362 for region: b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:54,752 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/5c6252080bbc48e6b6a023e08c36b716 for region: b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:54,772 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/de0cd278f5ae475c8e64f02e18179538 for region: b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:54,804 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/53ab7c6e85de44f29fcbaf368e39d276 for region: b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:54,816 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/488923b0b4ec41659120249ee07156c9 for region: b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:54,821 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/16a5e1fe22774dc782ac22bcb6a70060 for region: b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:54,828 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/13ee2b4b0c024358a690edb0b94aac79 for region: b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:54,836 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/5317f0b5b225487c82a9f79b071b429d for region: b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:54,843 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/8dd2e18ff3e64f1999a3bee1527292e3 for region: b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:54,844 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/912fddaa1bc14af3ad9378909904f17f for region: b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:54,846 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/a78695d99c44472fb73a8df5d283aed5 for region: b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:54,847 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/b94fd300967f43f7a6ca454238517875 for region: b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:54,847 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/8f0768252daf4fca8efafdcc6273b3c6 for region: b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:54,848 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/4d8bff8f40d4435e82f15595c5237edd for region: b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:54,848 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region b92c1c7e8ba3ec18a0b4e94867da9855 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-05-02 02:53:54,887 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-02 02:53:54,893 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-05-02 02:53:54,894 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-05-02 02:53:54,894 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-02 02:53:54,895 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:54,895 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-02 02:53:54,895 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-02 02:53:55,037 INFO [Listener at localhost/37119] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:55,037 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-02 02:53:55,037 INFO [Listener at localhost/37119] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-02 02:53:55,037 INFO [Listener at localhost/37119] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-02 02:53:55,038 ERROR [Listener at localhost/37119] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-02 02:53:55,039 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-02 02:53:55,039 INFO [Listener at localhost/37119] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-02 02:53:55,039 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-02 02:53:55,039 INFO [Listener at localhost/37119] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-02 02:53:55,039 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-02 02:53:55,041 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-05-02 02:53:55,041 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-02 02:53:55,041 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-02 02:53:55,041 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-02 02:53:55,041 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-02 02:53:55,042 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-02 02:53:55,042 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-02 02:53:55,042 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:53:55,043 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=084b1607f7ed8615120a04e4729fbafa, daughterA=ea7683cd3436a62eb16418a3c875d563, daughterB=408e891cab9b919f4038b233ba34779d 2023-05-02 02:53:55,043 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:53:55,043 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:53:55,043 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-02 02:53:55,044 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:55,044 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=b92c1c7e8ba3ec18a0b4e94867da9855, UNASSIGN 2023-05-02 02:53:55,044 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure b92c1c7e8ba3ec18a0b4e94867da9855, server=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:55,044 DEBUG [Listener at localhost/37119] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e held the lock before restarting, call acquireLock to restore it. 2023-05-02 02:53:55,044 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:55,044 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:55,045 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:55,045 DEBUG [Listener at localhost/37119] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=b92c1c7e8ba3ec18a0b4e94867da9855, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-02 02:53:55,045 DEBUG [Listener at localhost/37119] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure b92c1c7e8ba3ec18a0b4e94867da9855, server=jenkins-hbase4.apache.org,42019,1682996021676 didn't hold the lock before restarting, skip acquiring lock. 2023-05-02 02:53:55,045 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-05-02 02:53:55,045 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-02 02:53:55,046 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,38925,1682996021633, table=hbase:meta, region=1588230740 2023-05-02 02:53:55,046 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682996065046 2023-05-02 02:53:55,048 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:55,048 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(1517): Joining cluster... 2023-05-02 02:53:55,051 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=5c823d6f15ad7b8fe82f9ae4beee6513, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42019,1682996021676, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676, openSeqNum=2 2023-05-02 02:53:55,051 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=b92c1c7e8ba3ec18a0b4e94867da9855, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,42019,1682996021676, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676, openSeqNum=2 2023-05-02 02:53:55,051 INFO [Listener at localhost/37119] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-02 02:53:55,051 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682996095051 2023-05-02 02:53:55,051 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682996155051 2023-05-02 02:53:55,051 INFO [Listener at localhost/37119] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-02 02:53:55,055 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-02 02:53:55,057 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-02 02:53:55,057 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:55,057 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-02 02:53:55,057 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-02 02:53:55,302 WARN [Listener at localhost/37119] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-02 02:53:55,302 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-02 02:53:55,302 INFO [Listener at localhost/37119] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-02 02:53:55,302 INFO [Listener at localhost/37119] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-02 02:53:55,302 ERROR [Listener at localhost/37119] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-02 02:53:55,302 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-02 02:53:55,302 INFO [Listener at localhost/37119] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-02 02:53:55,302 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-02 02:53:55,302 INFO [Listener at localhost/37119] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-02 02:53:55,303 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-02 02:53:55,305 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-05-02 02:53:55,305 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-02 02:53:55,305 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-02 02:53:55,305 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-02 02:53:55,306 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-02 02:53:55,306 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-02 02:53:55,306 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-02 02:53:55,307 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:53:55,307 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=084b1607f7ed8615120a04e4729fbafa, daughterA=ea7683cd3436a62eb16418a3c875d563, daughterB=408e891cab9b919f4038b233ba34779d 2023-05-02 02:53:55,307 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:53:55,308 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:53:55,308 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-02 02:53:55,309 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:55,309 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=b92c1c7e8ba3ec18a0b4e94867da9855, UNASSIGN 2023-05-02 02:53:55,309 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure b92c1c7e8ba3ec18a0b4e94867da9855, server=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:55,309 DEBUG [Listener at localhost/37119] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e held the lock before restarting, call acquireLock to restore it. 2023-05-02 02:53:55,309 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:55,310 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:55,310 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:55,310 DEBUG [Listener at localhost/37119] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=b92c1c7e8ba3ec18a0b4e94867da9855, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-02 02:53:55,310 DEBUG [Listener at localhost/37119] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure b92c1c7e8ba3ec18a0b4e94867da9855, server=jenkins-hbase4.apache.org,42019,1682996021676 didn't hold the lock before restarting, skip acquiring lock. 2023-05-02 02:53:55,310 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-05-02 02:53:55,311 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-02 02:53:55,311 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,38925,1682996021633, table=hbase:meta, region=1588230740 2023-05-02 02:53:55,312 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682996065312 2023-05-02 02:53:55,313 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(1517): Joining cluster... 2023-05-02 02:53:55,313 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:55,316 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=5c823d6f15ad7b8fe82f9ae4beee6513, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42019,1682996021676, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676, openSeqNum=2 2023-05-02 02:53:55,316 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=b92c1c7e8ba3ec18a0b4e94867da9855, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,42019,1682996021676, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676, openSeqNum=2 2023-05-02 02:53:55,317 INFO [Listener at localhost/37119] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-02 02:53:55,317 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682996095317 2023-05-02 02:53:55,317 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682996155317 2023-05-02 02:53:55,317 INFO [Listener at localhost/37119] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-05-02 02:53:55,322 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1682996029879.b92c1c7e8ba3ec18a0b4e94867da9855.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1682996035321"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1682996035321"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1682996035321"}]},"ts":"1682996035321"} 2023-05-02 02:53:55,322 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1682996031910.c0e0b3bf0872704f385d48808df6a51e.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1682996035321"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682996035321"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682996035321"}]},"ts":"1682996035321"} 2023-05-02 02:53:55,322 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1682996031910.6e3e738dce7ec3e6ddbdadab93b4596e.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1682996035321"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682996035321"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682996035321"}]},"ts":"1682996035321"} 2023-05-02 02:53:55,328 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=c0e0b3bf0872704f385d48808df6a51e, ASSIGN}, {pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6e3e738dce7ec3e6ddbdadab93b4596e, ASSIGN}] 2023-05-02 02:53:55,330 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6e3e738dce7ec3e6ddbdadab93b4596e, ASSIGN 2023-05-02 02:53:55,331 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6e3e738dce7ec3e6ddbdadab93b4596e, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,42019,1682996021676; forceNewPlan=false, retain=false 2023-05-02 02:53:55,332 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=c0e0b3bf0872704f385d48808df6a51e, ASSIGN 2023-05-02 02:53:55,332 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=c0e0b3bf0872704f385d48808df6a51e, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,42019,1682996021676; forceNewPlan=false, retain=false 2023-05-02 02:53:55,481 INFO [jenkins-hbase4:41119] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-05-02 02:53:55,483 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=6e3e738dce7ec3e6ddbdadab93b4596e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:55,483 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1682996031910.6e3e738dce7ec3e6ddbdadab93b4596e.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1682996035483"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1682996035483"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682996035483"}]},"ts":"1682996035483"} 2023-05-02 02:53:55,486 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=32, state=RUNNABLE; OpenRegionProcedure 6e3e738dce7ec3e6ddbdadab93b4596e, server=jenkins-hbase4.apache.org,42019,1682996021676}] 2023-05-02 02:53:55,489 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=c0e0b3bf0872704f385d48808df6a51e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:55,489 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1682996031910.c0e0b3bf0872704f385d48808df6a51e.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1682996035489"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1682996035489"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682996035489"}]},"ts":"1682996035489"} 2023-05-02 02:53:55,491 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=34, ppid=31, state=RUNNABLE; OpenRegionProcedure c0e0b3bf0872704f385d48808df6a51e, server=jenkins-hbase4.apache.org,42019,1682996021676}] 2023-05-02 02:53:55,642 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitWithoutPONR,,1682996031910.c0e0b3bf0872704f385d48808df6a51e. 2023-05-02 02:53:55,642 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => c0e0b3bf0872704f385d48808df6a51e, NAME => 'testSplitWithoutPONR,,1682996031910.c0e0b3bf0872704f385d48808df6a51e.', STARTKEY => '', ENDKEY => '41'} 2023-05-02 02:53:55,642 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-02 02:53:55,642 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR c0e0b3bf0872704f385d48808df6a51e 2023-05-02 02:53:55,643 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitWithoutPONR,,1682996031910.c0e0b3bf0872704f385d48808df6a51e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-02 02:53:55,643 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for c0e0b3bf0872704f385d48808df6a51e 2023-05-02 02:53:55,643 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for c0e0b3bf0872704f385d48808df6a51e 2023-05-02 02:53:55,644 INFO [StoreOpener-c0e0b3bf0872704f385d48808df6a51e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region c0e0b3bf0872704f385d48808df6a51e 2023-05-02 02:53:55,645 DEBUG [StoreOpener-c0e0b3bf0872704f385d48808df6a51e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf1 2023-05-02 02:53:55,645 DEBUG [StoreOpener-c0e0b3bf0872704f385d48808df6a51e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf1 2023-05-02 02:53:55,646 INFO [StoreOpener-c0e0b3bf0872704f385d48808df6a51e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c0e0b3bf0872704f385d48808df6a51e columnFamilyName cf1 2023-05-02 02:53:55,661 DEBUG [StoreOpener-c0e0b3bf0872704f385d48808df6a51e-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf1/13ee2b4b0c024358a690edb0b94aac79.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/13ee2b4b0c024358a690edb0b94aac79-bottom 2023-05-02 02:53:55,666 DEBUG [StoreOpener-c0e0b3bf0872704f385d48808df6a51e-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf1/16a5e1fe22774dc782ac22bcb6a70060.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/16a5e1fe22774dc782ac22bcb6a70060-bottom 2023-05-02 02:53:55,671 DEBUG [StoreOpener-c0e0b3bf0872704f385d48808df6a51e-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf1/4d8bff8f40d4435e82f15595c5237edd.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/4d8bff8f40d4435e82f15595c5237edd-bottom 2023-05-02 02:53:55,677 DEBUG [StoreOpener-c0e0b3bf0872704f385d48808df6a51e-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf1/5317f0b5b225487c82a9f79b071b429d.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/5317f0b5b225487c82a9f79b071b429d-bottom 2023-05-02 02:53:55,683 DEBUG [StoreOpener-c0e0b3bf0872704f385d48808df6a51e-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf1/8dd2e18ff3e64f1999a3bee1527292e3.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/8dd2e18ff3e64f1999a3bee1527292e3-bottom 2023-05-02 02:53:55,688 DEBUG [StoreOpener-c0e0b3bf0872704f385d48808df6a51e-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf1/a78695d99c44472fb73a8df5d283aed5.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/a78695d99c44472fb73a8df5d283aed5-bottom 2023-05-02 02:53:55,693 DEBUG [StoreOpener-c0e0b3bf0872704f385d48808df6a51e-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf1/de0cd278f5ae475c8e64f02e18179538.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/de0cd278f5ae475c8e64f02e18179538-bottom 2023-05-02 02:53:55,694 INFO [StoreOpener-c0e0b3bf0872704f385d48808df6a51e-1] regionserver.HStore(310): Store=c0e0b3bf0872704f385d48808df6a51e/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-02 02:53:55,694 INFO [StoreOpener-c0e0b3bf0872704f385d48808df6a51e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region c0e0b3bf0872704f385d48808df6a51e 2023-05-02 02:53:55,695 DEBUG [StoreOpener-c0e0b3bf0872704f385d48808df6a51e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf2 2023-05-02 02:53:55,695 DEBUG [StoreOpener-c0e0b3bf0872704f385d48808df6a51e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf2 2023-05-02 02:53:55,695 INFO [StoreOpener-c0e0b3bf0872704f385d48808df6a51e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c0e0b3bf0872704f385d48808df6a51e columnFamilyName cf2 2023-05-02 02:53:55,711 DEBUG [StoreOpener-c0e0b3bf0872704f385d48808df6a51e-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf2/427ba387297e4b318fda28a69d090362.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/427ba387297e4b318fda28a69d090362-bottom 2023-05-02 02:53:55,716 DEBUG [StoreOpener-c0e0b3bf0872704f385d48808df6a51e-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf2/488923b0b4ec41659120249ee07156c9.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/488923b0b4ec41659120249ee07156c9-bottom 2023-05-02 02:53:55,721 DEBUG [StoreOpener-c0e0b3bf0872704f385d48808df6a51e-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf2/53ab7c6e85de44f29fcbaf368e39d276.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/53ab7c6e85de44f29fcbaf368e39d276-bottom 2023-05-02 02:53:55,725 DEBUG [StoreOpener-c0e0b3bf0872704f385d48808df6a51e-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf2/5c6252080bbc48e6b6a023e08c36b716.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/5c6252080bbc48e6b6a023e08c36b716-bottom 2023-05-02 02:53:55,730 DEBUG [StoreOpener-c0e0b3bf0872704f385d48808df6a51e-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf2/8f0768252daf4fca8efafdcc6273b3c6.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/8f0768252daf4fca8efafdcc6273b3c6-bottom 2023-05-02 02:53:55,735 DEBUG [StoreOpener-c0e0b3bf0872704f385d48808df6a51e-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf2/912fddaa1bc14af3ad9378909904f17f.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/912fddaa1bc14af3ad9378909904f17f-bottom 2023-05-02 02:53:55,739 DEBUG [StoreOpener-c0e0b3bf0872704f385d48808df6a51e-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf2/b94fd300967f43f7a6ca454238517875.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/b94fd300967f43f7a6ca454238517875-bottom 2023-05-02 02:53:55,740 INFO [StoreOpener-c0e0b3bf0872704f385d48808df6a51e-1] regionserver.HStore(310): Store=c0e0b3bf0872704f385d48808df6a51e/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-02 02:53:55,741 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e 2023-05-02 02:53:55,741 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e 2023-05-02 02:53:55,743 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-02 02:53:55,745 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for c0e0b3bf0872704f385d48808df6a51e 2023-05-02 02:53:55,746 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened c0e0b3bf0872704f385d48808df6a51e; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71955643, jitterRate=0.07222263514995575}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-02 02:53:55,746 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for c0e0b3bf0872704f385d48808df6a51e: 2023-05-02 02:53:55,747 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,,1682996031910.c0e0b3bf0872704f385d48808df6a51e., pid=34, masterSystemTime=1682996035638 2023-05-02 02:53:55,747 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-02 02:53:55,748 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-05-02 02:53:55,749 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-02 02:53:55,748 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-02 02:53:55,749 INFO [RS:2;jenkins-hbase4:42019-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,,1682996031910.c0e0b3bf0872704f385d48808df6a51e. 2023-05-02 02:53:55,749 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] regionserver.HStore(1912): c0e0b3bf0872704f385d48808df6a51e/cf2 is initiating minor compaction (all files) 2023-05-02 02:53:55,749 INFO [RS:2;jenkins-hbase4:42019-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,,1682996031910.c0e0b3bf0872704f385d48808df6a51e. 2023-05-02 02:53:55,749 INFO [RS:2;jenkins-hbase4:42019-longCompactions-0] regionserver.HRegion(2259): Starting compaction of c0e0b3bf0872704f385d48808df6a51e/cf2 in testSplitWithoutPONR,,1682996031910.c0e0b3bf0872704f385d48808df6a51e. 2023-05-02 02:53:55,749 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] regionserver.HStore(1912): c0e0b3bf0872704f385d48808df6a51e/cf1 is initiating minor compaction (all files) 2023-05-02 02:53:55,749 INFO [RS:2;jenkins-hbase4:42019-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of c0e0b3bf0872704f385d48808df6a51e/cf1 in testSplitWithoutPONR,,1682996031910.c0e0b3bf0872704f385d48808df6a51e. 2023-05-02 02:53:55,749 INFO [RS:2;jenkins-hbase4:42019-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf2/b94fd300967f43f7a6ca454238517875.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/b94fd300967f43f7a6ca454238517875-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf2/8f0768252daf4fca8efafdcc6273b3c6.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/8f0768252daf4fca8efafdcc6273b3c6-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf2/5c6252080bbc48e6b6a023e08c36b716.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/5c6252080bbc48e6b6a023e08c36b716-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf2/488923b0b4ec41659120249ee07156c9.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/488923b0b4ec41659120249ee07156c9-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf2/912fddaa1bc14af3ad9378909904f17f.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/912fddaa1bc14af3ad9378909904f17f-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf2/53ab7c6e85de44f29fcbaf368e39d276.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/53ab7c6e85de44f29fcbaf368e39d276-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf2/427ba387297e4b318fda28a69d090362.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/427ba387297e4b318fda28a69d090362-bottom] into tmpdir=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/.tmp, totalSize=34.9 K 2023-05-02 02:53:55,749 INFO [RS:2;jenkins-hbase4:42019-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf1/5317f0b5b225487c82a9f79b071b429d.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/5317f0b5b225487c82a9f79b071b429d-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf1/a78695d99c44472fb73a8df5d283aed5.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/a78695d99c44472fb73a8df5d283aed5-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf1/de0cd278f5ae475c8e64f02e18179538.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/de0cd278f5ae475c8e64f02e18179538-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf1/16a5e1fe22774dc782ac22bcb6a70060.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/16a5e1fe22774dc782ac22bcb6a70060-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf1/13ee2b4b0c024358a690edb0b94aac79.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/13ee2b4b0c024358a690edb0b94aac79-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf1/4d8bff8f40d4435e82f15595c5237edd.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/4d8bff8f40d4435e82f15595c5237edd-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf1/8dd2e18ff3e64f1999a3bee1527292e3.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/8dd2e18ff3e64f1999a3bee1527292e3-bottom] into tmpdir=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/.tmp, totalSize=34.9 K 2023-05-02 02:53:55,750 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] compactions.Compactor(207): Compacting b94fd300967f43f7a6ca454238517875.b92c1c7e8ba3ec18a0b4e94867da9855, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1682996030390 2023-05-02 02:53:55,750 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] compactions.Compactor(207): Compacting 5317f0b5b225487c82a9f79b071b429d.b92c1c7e8ba3ec18a0b4e94867da9855, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1682996030390 2023-05-02 02:53:55,750 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,,1682996031910.c0e0b3bf0872704f385d48808df6a51e. 2023-05-02 02:53:55,751 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitWithoutPONR,,1682996031910.c0e0b3bf0872704f385d48808df6a51e. 2023-05-02 02:53:55,751 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitWithoutPONR,41,1682996031910.6e3e738dce7ec3e6ddbdadab93b4596e. 2023-05-02 02:53:55,751 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] compactions.Compactor(207): Compacting 8f0768252daf4fca8efafdcc6273b3c6.b92c1c7e8ba3ec18a0b4e94867da9855, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1682996030603 2023-05-02 02:53:55,751 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] compactions.Compactor(207): Compacting a78695d99c44472fb73a8df5d283aed5.b92c1c7e8ba3ec18a0b4e94867da9855, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1682996030603 2023-05-02 02:53:55,751 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 6e3e738dce7ec3e6ddbdadab93b4596e, NAME => 'testSplitWithoutPONR,41,1682996031910.6e3e738dce7ec3e6ddbdadab93b4596e.', STARTKEY => '41', ENDKEY => ''} 2023-05-02 02:53:55,751 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=c0e0b3bf0872704f385d48808df6a51e, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:55,751 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] compactions.Compactor(207): Compacting 5c6252080bbc48e6b6a023e08c36b716.b92c1c7e8ba3ec18a0b4e94867da9855, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1682996030826 2023-05-02 02:53:55,751 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-02 02:53:55,751 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1682996031910.c0e0b3bf0872704f385d48808df6a51e.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1682996035751"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1682996035751"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1682996035751"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682996035751"}]},"ts":"1682996035751"} 2023-05-02 02:53:55,751 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] compactions.Compactor(207): Compacting de0cd278f5ae475c8e64f02e18179538.b92c1c7e8ba3ec18a0b4e94867da9855, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1682996030826 2023-05-02 02:53:55,752 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:55,752 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitWithoutPONR,41,1682996031910.6e3e738dce7ec3e6ddbdadab93b4596e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-02 02:53:55,752 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] compactions.Compactor(207): Compacting 488923b0b4ec41659120249ee07156c9.b92c1c7e8ba3ec18a0b4e94867da9855, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1682996031033 2023-05-02 02:53:55,752 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:55,752 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:55,752 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] compactions.Compactor(207): Compacting 16a5e1fe22774dc782ac22bcb6a70060.b92c1c7e8ba3ec18a0b4e94867da9855, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1682996031033 2023-05-02 02:53:55,752 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] compactions.Compactor(207): Compacting 912fddaa1bc14af3ad9378909904f17f.b92c1c7e8ba3ec18a0b4e94867da9855, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1682996031235 2023-05-02 02:53:55,753 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] compactions.Compactor(207): Compacting 13ee2b4b0c024358a690edb0b94aac79.b92c1c7e8ba3ec18a0b4e94867da9855, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1682996031235 2023-05-02 02:53:55,753 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] compactions.Compactor(207): Compacting 53ab7c6e85de44f29fcbaf368e39d276.b92c1c7e8ba3ec18a0b4e94867da9855, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1682996031444 2023-05-02 02:53:55,754 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] compactions.Compactor(207): Compacting 4d8bff8f40d4435e82f15595c5237edd.b92c1c7e8ba3ec18a0b4e94867da9855, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1682996031444 2023-05-02 02:53:55,754 INFO [StoreOpener-6e3e738dce7ec3e6ddbdadab93b4596e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:55,754 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] compactions.Compactor(207): Compacting 427ba387297e4b318fda28a69d090362.b92c1c7e8ba3ec18a0b4e94867da9855, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1682996031647 2023-05-02 02:53:55,754 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] compactions.Compactor(207): Compacting 8dd2e18ff3e64f1999a3bee1527292e3.b92c1c7e8ba3ec18a0b4e94867da9855, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1682996031647 2023-05-02 02:53:55,756 DEBUG [StoreOpener-6e3e738dce7ec3e6ddbdadab93b4596e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf1 2023-05-02 02:53:55,756 DEBUG [StoreOpener-6e3e738dce7ec3e6ddbdadab93b4596e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf1 2023-05-02 02:53:55,756 INFO [StoreOpener-6e3e738dce7ec3e6ddbdadab93b4596e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6e3e738dce7ec3e6ddbdadab93b4596e columnFamilyName cf1 2023-05-02 02:53:55,756 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=34, resume processing ppid=31 2023-05-02 02:53:55,757 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=34, ppid=31, state=SUCCESS; OpenRegionProcedure c0e0b3bf0872704f385d48808df6a51e, server=jenkins-hbase4.apache.org,42019,1682996021676 in 262 msec 2023-05-02 02:53:55,759 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=c0e0b3bf0872704f385d48808df6a51e, ASSIGN in 431 msec 2023-05-02 02:53:55,773 INFO [RS:2;jenkins-hbase4:42019-longCompactions-0] throttle.PressureAwareThroughputController(145): c0e0b3bf0872704f385d48808df6a51e#cf2#compaction#32 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-05-02 02:53:55,773 INFO [RS:2;jenkins-hbase4:42019-shortCompactions-0] throttle.PressureAwareThroughputController(145): c0e0b3bf0872704f385d48808df6a51e#cf1#compaction#33 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-02 02:53:55,776 DEBUG [StoreOpener-6e3e738dce7ec3e6ddbdadab93b4596e-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf1/13ee2b4b0c024358a690edb0b94aac79.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/13ee2b4b0c024358a690edb0b94aac79-top 2023-05-02 02:53:55,787 DEBUG [StoreOpener-6e3e738dce7ec3e6ddbdadab93b4596e-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf1/16a5e1fe22774dc782ac22bcb6a70060.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/16a5e1fe22774dc782ac22bcb6a70060-top 2023-05-02 02:53:55,797 DEBUG [StoreOpener-6e3e738dce7ec3e6ddbdadab93b4596e-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf1/4d8bff8f40d4435e82f15595c5237edd.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/4d8bff8f40d4435e82f15595c5237edd-top 2023-05-02 02:53:55,801 DEBUG [StoreOpener-6e3e738dce7ec3e6ddbdadab93b4596e-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf1/5317f0b5b225487c82a9f79b071b429d.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/5317f0b5b225487c82a9f79b071b429d-top 2023-05-02 02:53:55,805 DEBUG [StoreOpener-6e3e738dce7ec3e6ddbdadab93b4596e-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf1/8dd2e18ff3e64f1999a3bee1527292e3.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/8dd2e18ff3e64f1999a3bee1527292e3-top 2023-05-02 02:53:55,809 DEBUG [StoreOpener-6e3e738dce7ec3e6ddbdadab93b4596e-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf1/a78695d99c44472fb73a8df5d283aed5.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/a78695d99c44472fb73a8df5d283aed5-top 2023-05-02 02:53:55,817 DEBUG [StoreOpener-6e3e738dce7ec3e6ddbdadab93b4596e-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf1/de0cd278f5ae475c8e64f02e18179538.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/de0cd278f5ae475c8e64f02e18179538-top 2023-05-02 02:53:55,817 INFO [StoreOpener-6e3e738dce7ec3e6ddbdadab93b4596e-1] regionserver.HStore(310): Store=6e3e738dce7ec3e6ddbdadab93b4596e/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-02 02:53:55,817 INFO [StoreOpener-6e3e738dce7ec3e6ddbdadab93b4596e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:55,817 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/.tmp/cf2/e9d8bbfb5ab04c19a12d8424d2d6c45c as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf2/e9d8bbfb5ab04c19a12d8424d2d6c45c 2023-05-02 02:53:55,818 DEBUG [StoreOpener-6e3e738dce7ec3e6ddbdadab93b4596e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf2 2023-05-02 02:53:55,818 DEBUG [StoreOpener-6e3e738dce7ec3e6ddbdadab93b4596e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf2 2023-05-02 02:53:55,819 INFO [StoreOpener-6e3e738dce7ec3e6ddbdadab93b4596e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6e3e738dce7ec3e6ddbdadab93b4596e columnFamilyName cf2 2023-05-02 02:53:55,819 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/.tmp/cf1/a9b141707bf14943b1f5a27a40c906c0 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf1/a9b141707bf14943b1f5a27a40c906c0 2023-05-02 02:53:55,828 INFO [RS:2;jenkins-hbase4:42019-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in c0e0b3bf0872704f385d48808df6a51e/cf2 of c0e0b3bf0872704f385d48808df6a51e into e9d8bbfb5ab04c19a12d8424d2d6c45c(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-02 02:53:55,829 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for c0e0b3bf0872704f385d48808df6a51e: 2023-05-02 02:53:55,829 INFO [RS:2;jenkins-hbase4:42019-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1682996031910.c0e0b3bf0872704f385d48808df6a51e., storeName=c0e0b3bf0872704f385d48808df6a51e/cf2, priority=9, startTime=1682996035748; duration=0sec 2023-05-02 02:53:55,829 INFO [RS:2;jenkins-hbase4:42019-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in c0e0b3bf0872704f385d48808df6a51e/cf1 of c0e0b3bf0872704f385d48808df6a51e into a9b141707bf14943b1f5a27a40c906c0(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-02 02:53:55,829 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for c0e0b3bf0872704f385d48808df6a51e: 2023-05-02 02:53:55,829 INFO [RS:2;jenkins-hbase4:42019-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1682996031910.c0e0b3bf0872704f385d48808df6a51e., storeName=c0e0b3bf0872704f385d48808df6a51e/cf1, priority=9, startTime=1682996035747; duration=0sec 2023-05-02 02:53:55,829 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-02 02:53:55,829 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-02 02:53:55,843 DEBUG [StoreOpener-6e3e738dce7ec3e6ddbdadab93b4596e-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf2/427ba387297e4b318fda28a69d090362.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/427ba387297e4b318fda28a69d090362-top 2023-05-02 02:53:55,853 DEBUG [StoreOpener-6e3e738dce7ec3e6ddbdadab93b4596e-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf2/488923b0b4ec41659120249ee07156c9.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/488923b0b4ec41659120249ee07156c9-top 2023-05-02 02:53:55,857 DEBUG [StoreOpener-6e3e738dce7ec3e6ddbdadab93b4596e-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf2/53ab7c6e85de44f29fcbaf368e39d276.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/53ab7c6e85de44f29fcbaf368e39d276-top 2023-05-02 02:53:55,861 DEBUG [StoreOpener-6e3e738dce7ec3e6ddbdadab93b4596e-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf2/5c6252080bbc48e6b6a023e08c36b716.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/5c6252080bbc48e6b6a023e08c36b716-top 2023-05-02 02:53:55,865 DEBUG [StoreOpener-6e3e738dce7ec3e6ddbdadab93b4596e-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf2/8f0768252daf4fca8efafdcc6273b3c6.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/8f0768252daf4fca8efafdcc6273b3c6-top 2023-05-02 02:53:55,870 DEBUG [StoreOpener-6e3e738dce7ec3e6ddbdadab93b4596e-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf2/912fddaa1bc14af3ad9378909904f17f.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/912fddaa1bc14af3ad9378909904f17f-top 2023-05-02 02:53:55,874 DEBUG [StoreOpener-6e3e738dce7ec3e6ddbdadab93b4596e-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf2/b94fd300967f43f7a6ca454238517875.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/b94fd300967f43f7a6ca454238517875-top 2023-05-02 02:53:55,874 INFO [StoreOpener-6e3e738dce7ec3e6ddbdadab93b4596e-1] regionserver.HStore(310): Store=6e3e738dce7ec3e6ddbdadab93b4596e/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-02 02:53:55,875 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:55,876 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:55,878 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-02 02:53:55,879 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:55,880 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 6e3e738dce7ec3e6ddbdadab93b4596e; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69176420, jitterRate=0.030808985233306885}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-02 02:53:55,880 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 6e3e738dce7ec3e6ddbdadab93b4596e: 2023-05-02 02:53:55,881 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,41,1682996031910.6e3e738dce7ec3e6ddbdadab93b4596e., pid=33, masterSystemTime=1682996035638 2023-05-02 02:53:55,881 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-02 02:53:55,882 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-02 02:53:55,882 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-02 02:53:55,882 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-02 02:53:55,883 INFO [RS:2;jenkins-hbase4:42019-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,41,1682996031910.6e3e738dce7ec3e6ddbdadab93b4596e. 2023-05-02 02:53:55,883 INFO [RS:2;jenkins-hbase4:42019-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,41,1682996031910.6e3e738dce7ec3e6ddbdadab93b4596e. 2023-05-02 02:53:55,883 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] regionserver.HStore(1912): 6e3e738dce7ec3e6ddbdadab93b4596e/cf2 is initiating minor compaction (all files) 2023-05-02 02:53:55,883 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] regionserver.HStore(1912): 6e3e738dce7ec3e6ddbdadab93b4596e/cf1 is initiating minor compaction (all files) 2023-05-02 02:53:55,883 INFO [RS:2;jenkins-hbase4:42019-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 6e3e738dce7ec3e6ddbdadab93b4596e/cf2 in testSplitWithoutPONR,41,1682996031910.6e3e738dce7ec3e6ddbdadab93b4596e. 2023-05-02 02:53:55,883 INFO [RS:2;jenkins-hbase4:42019-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 6e3e738dce7ec3e6ddbdadab93b4596e/cf1 in testSplitWithoutPONR,41,1682996031910.6e3e738dce7ec3e6ddbdadab93b4596e. 2023-05-02 02:53:55,883 INFO [RS:2;jenkins-hbase4:42019-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf1/5317f0b5b225487c82a9f79b071b429d.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/5317f0b5b225487c82a9f79b071b429d-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf1/a78695d99c44472fb73a8df5d283aed5.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/a78695d99c44472fb73a8df5d283aed5-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf1/de0cd278f5ae475c8e64f02e18179538.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/de0cd278f5ae475c8e64f02e18179538-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf1/16a5e1fe22774dc782ac22bcb6a70060.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/16a5e1fe22774dc782ac22bcb6a70060-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf1/13ee2b4b0c024358a690edb0b94aac79.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/13ee2b4b0c024358a690edb0b94aac79-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf1/4d8bff8f40d4435e82f15595c5237edd.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/4d8bff8f40d4435e82f15595c5237edd-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf1/8dd2e18ff3e64f1999a3bee1527292e3.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/8dd2e18ff3e64f1999a3bee1527292e3-top] into tmpdir=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/.tmp, totalSize=34.9 K 2023-05-02 02:53:55,883 INFO [RS:2;jenkins-hbase4:42019-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf2/b94fd300967f43f7a6ca454238517875.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/b94fd300967f43f7a6ca454238517875-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf2/8f0768252daf4fca8efafdcc6273b3c6.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/8f0768252daf4fca8efafdcc6273b3c6-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf2/5c6252080bbc48e6b6a023e08c36b716.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/5c6252080bbc48e6b6a023e08c36b716-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf2/488923b0b4ec41659120249ee07156c9.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/488923b0b4ec41659120249ee07156c9-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf2/912fddaa1bc14af3ad9378909904f17f.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/912fddaa1bc14af3ad9378909904f17f-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf2/53ab7c6e85de44f29fcbaf368e39d276.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/53ab7c6e85de44f29fcbaf368e39d276-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf2/427ba387297e4b318fda28a69d090362.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/427ba387297e4b318fda28a69d090362-top] into tmpdir=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/.tmp, totalSize=34.9 K 2023-05-02 02:53:55,884 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] compactions.Compactor(207): Compacting 5317f0b5b225487c82a9f79b071b429d.b92c1c7e8ba3ec18a0b4e94867da9855, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1682996030390 2023-05-02 02:53:55,884 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] compactions.Compactor(207): Compacting b94fd300967f43f7a6ca454238517875.b92c1c7e8ba3ec18a0b4e94867da9855, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1682996030390 2023-05-02 02:53:55,885 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,41,1682996031910.6e3e738dce7ec3e6ddbdadab93b4596e. 2023-05-02 02:53:55,885 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitWithoutPONR,41,1682996031910.6e3e738dce7ec3e6ddbdadab93b4596e. 2023-05-02 02:53:55,885 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] compactions.Compactor(207): Compacting a78695d99c44472fb73a8df5d283aed5.b92c1c7e8ba3ec18a0b4e94867da9855, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1682996030603 2023-05-02 02:53:55,885 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] compactions.Compactor(207): Compacting 8f0768252daf4fca8efafdcc6273b3c6.b92c1c7e8ba3ec18a0b4e94867da9855, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1682996030603 2023-05-02 02:53:55,885 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=6e3e738dce7ec3e6ddbdadab93b4596e, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:55,885 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] compactions.Compactor(207): Compacting de0cd278f5ae475c8e64f02e18179538.b92c1c7e8ba3ec18a0b4e94867da9855, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1682996030826 2023-05-02 02:53:55,886 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,41,1682996031910.6e3e738dce7ec3e6ddbdadab93b4596e.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1682996035885"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1682996035885"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1682996035885"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682996035885"}]},"ts":"1682996035885"} 2023-05-02 02:53:55,886 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] compactions.Compactor(207): Compacting 5c6252080bbc48e6b6a023e08c36b716.b92c1c7e8ba3ec18a0b4e94867da9855, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1682996030826 2023-05-02 02:53:55,886 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] compactions.Compactor(207): Compacting 16a5e1fe22774dc782ac22bcb6a70060.b92c1c7e8ba3ec18a0b4e94867da9855, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1682996031033 2023-05-02 02:53:55,886 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] compactions.Compactor(207): Compacting 488923b0b4ec41659120249ee07156c9.b92c1c7e8ba3ec18a0b4e94867da9855, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1682996031033 2023-05-02 02:53:55,887 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] compactions.Compactor(207): Compacting 912fddaa1bc14af3ad9378909904f17f.b92c1c7e8ba3ec18a0b4e94867da9855, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1682996031235 2023-05-02 02:53:55,887 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] compactions.Compactor(207): Compacting 13ee2b4b0c024358a690edb0b94aac79.b92c1c7e8ba3ec18a0b4e94867da9855, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1682996031235 2023-05-02 02:53:55,888 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] compactions.Compactor(207): Compacting 53ab7c6e85de44f29fcbaf368e39d276.b92c1c7e8ba3ec18a0b4e94867da9855, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1682996031444 2023-05-02 02:53:55,888 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] compactions.Compactor(207): Compacting 4d8bff8f40d4435e82f15595c5237edd.b92c1c7e8ba3ec18a0b4e94867da9855, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1682996031444 2023-05-02 02:53:55,888 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] compactions.Compactor(207): Compacting 427ba387297e4b318fda28a69d090362.b92c1c7e8ba3ec18a0b4e94867da9855, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1682996031647 2023-05-02 02:53:55,889 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] compactions.Compactor(207): Compacting 8dd2e18ff3e64f1999a3bee1527292e3.b92c1c7e8ba3ec18a0b4e94867da9855, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1682996031647 2023-05-02 02:53:55,890 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=32 2023-05-02 02:53:55,890 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=32, state=SUCCESS; OpenRegionProcedure 6e3e738dce7ec3e6ddbdadab93b4596e, server=jenkins-hbase4.apache.org,42019,1682996021676 in 401 msec 2023-05-02 02:53:55,894 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=32, resume processing ppid=28 2023-05-02 02:53:55,894 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6e3e738dce7ec3e6ddbdadab93b4596e, ASSIGN in 565 msec 2023-05-02 02:53:55,896 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e in 3.9840 sec 2023-05-02 02:53:55,904 INFO [RS:2;jenkins-hbase4:42019-longCompactions-0] throttle.PressureAwareThroughputController(145): 6e3e738dce7ec3e6ddbdadab93b4596e#cf2#compaction#34 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-02 02:53:55,912 INFO [RS:2;jenkins-hbase4:42019-shortCompactions-0] throttle.PressureAwareThroughputController(145): 6e3e738dce7ec3e6ddbdadab93b4596e#cf1#compaction#35 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-02 02:53:55,928 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/.tmp/cf2/806fb6b1551f4e418ab2610151e25f35 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf2/806fb6b1551f4e418ab2610151e25f35 2023-05-02 02:53:55,930 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/.tmp/cf1/ce982974d1bc4e85b6890d1cdeffeb2b as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf1/ce982974d1bc4e85b6890d1cdeffeb2b 2023-05-02 02:53:55,935 INFO [RS:2;jenkins-hbase4:42019-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 6e3e738dce7ec3e6ddbdadab93b4596e/cf1 of 6e3e738dce7ec3e6ddbdadab93b4596e into ce982974d1bc4e85b6890d1cdeffeb2b(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-02 02:53:55,935 INFO [RS:2;jenkins-hbase4:42019-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 6e3e738dce7ec3e6ddbdadab93b4596e/cf2 of 6e3e738dce7ec3e6ddbdadab93b4596e into 806fb6b1551f4e418ab2610151e25f35(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-02 02:53:55,935 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 6e3e738dce7ec3e6ddbdadab93b4596e: 2023-05-02 02:53:55,936 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 6e3e738dce7ec3e6ddbdadab93b4596e: 2023-05-02 02:53:55,936 INFO [RS:2;jenkins-hbase4:42019-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1682996031910.6e3e738dce7ec3e6ddbdadab93b4596e., storeName=6e3e738dce7ec3e6ddbdadab93b4596e/cf1, priority=9, startTime=1682996035881; duration=0sec 2023-05-02 02:53:55,936 INFO [RS:2;jenkins-hbase4:42019-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1682996031910.6e3e738dce7ec3e6ddbdadab93b4596e., storeName=6e3e738dce7ec3e6ddbdadab93b4596e/cf2, priority=9, startTime=1682996035882; duration=0sec 2023-05-02 02:53:55,936 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-02 02:53:55,936 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-02 02:53:56,068 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42019] ipc.CallRunner(144): callId: 239 service: ClientService methodName: Scan size: 143 connection: 172.31.14.131:58782 deadline: 1682996096068, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitWithoutPONR,,1682996029879.b92c1c7e8ba3ec18a0b4e94867da9855. is not online on jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:56,181 DEBUG [Listener at localhost/37119] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-05-02 02:53:56,189 INFO [Listener at localhost/37119] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-05-02 02:53:56,191 DEBUG [Listener at localhost/37119] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-05-02 02:53:56,207 WARN [Listener at localhost/37119] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-02 02:53:56,209 INFO [Listener at localhost/37119] client.HBaseAdmin$15(890): Started disable of testSplitWithoutPONR 2023-05-02 02:53:56,209 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testSplitWithoutPONR 2023-05-02 02:53:56,210 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure2.ProcedureExecutor(1029): Stored pid=35, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitWithoutPONR 2023-05-02 02:53:56,213 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-05-02 02:53:56,214 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682996036214"}]},"ts":"1682996036214"} 2023-05-02 02:53:56,215 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLING in hbase:meta 2023-05-02 02:53:56,217 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitWithoutPONR to state=DISABLING 2023-05-02 02:53:56,218 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=c0e0b3bf0872704f385d48808df6a51e, UNASSIGN}, {pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6e3e738dce7ec3e6ddbdadab93b4596e, UNASSIGN}] 2023-05-02 02:53:56,219 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6e3e738dce7ec3e6ddbdadab93b4596e, UNASSIGN 2023-05-02 02:53:56,219 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=6e3e738dce7ec3e6ddbdadab93b4596e, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:56,220 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1682996031910.6e3e738dce7ec3e6ddbdadab93b4596e.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1682996036219"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1682996036219"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682996036219"}]},"ts":"1682996036219"} 2023-05-02 02:53:56,221 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=38, ppid=37, state=RUNNABLE; CloseRegionProcedure 6e3e738dce7ec3e6ddbdadab93b4596e, server=jenkins-hbase4.apache.org,42019,1682996021676}] 2023-05-02 02:53:56,224 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=c0e0b3bf0872704f385d48808df6a51e, UNASSIGN 2023-05-02 02:53:56,225 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=c0e0b3bf0872704f385d48808df6a51e, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:56,225 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1682996031910.c0e0b3bf0872704f385d48808df6a51e.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1682996036225"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1682996036225"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682996036225"}]},"ts":"1682996036225"} 2023-05-02 02:53:56,226 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=39, ppid=36, state=RUNNABLE; CloseRegionProcedure c0e0b3bf0872704f385d48808df6a51e, server=jenkins-hbase4.apache.org,42019,1682996021676}] 2023-05-02 02:53:56,314 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-05-02 02:53:56,373 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close c0e0b3bf0872704f385d48808df6a51e 2023-05-02 02:53:56,374 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing c0e0b3bf0872704f385d48808df6a51e, disabling compactions & flushes 2023-05-02 02:53:56,374 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitWithoutPONR,,1682996031910.c0e0b3bf0872704f385d48808df6a51e. 2023-05-02 02:53:56,374 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitWithoutPONR,,1682996031910.c0e0b3bf0872704f385d48808df6a51e. 2023-05-02 02:53:56,374 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitWithoutPONR,,1682996031910.c0e0b3bf0872704f385d48808df6a51e. after waiting 0 ms 2023-05-02 02:53:56,374 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitWithoutPONR,,1682996031910.c0e0b3bf0872704f385d48808df6a51e. 2023-05-02 02:53:56,385 DEBUG [StoreCloser-testSplitWithoutPONR,,1682996031910.c0e0b3bf0872704f385d48808df6a51e.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf1/5317f0b5b225487c82a9f79b071b429d.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/5317f0b5b225487c82a9f79b071b429d-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf1/a78695d99c44472fb73a8df5d283aed5.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/a78695d99c44472fb73a8df5d283aed5-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf1/de0cd278f5ae475c8e64f02e18179538.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/de0cd278f5ae475c8e64f02e18179538-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf1/16a5e1fe22774dc782ac22bcb6a70060.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/16a5e1fe22774dc782ac22bcb6a70060-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf1/13ee2b4b0c024358a690edb0b94aac79.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/13ee2b4b0c024358a690edb0b94aac79-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf1/4d8bff8f40d4435e82f15595c5237edd.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/4d8bff8f40d4435e82f15595c5237edd-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf1/8dd2e18ff3e64f1999a3bee1527292e3.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/8dd2e18ff3e64f1999a3bee1527292e3-bottom] to archive 2023-05-02 02:53:56,387 DEBUG [StoreCloser-testSplitWithoutPONR,,1682996031910.c0e0b3bf0872704f385d48808df6a51e.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-02 02:53:56,389 DEBUG [StoreCloser-testSplitWithoutPONR,,1682996031910.c0e0b3bf0872704f385d48808df6a51e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf1/5317f0b5b225487c82a9f79b071b429d.b92c1c7e8ba3ec18a0b4e94867da9855 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf1/5317f0b5b225487c82a9f79b071b429d.b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:56,390 DEBUG [StoreCloser-testSplitWithoutPONR,,1682996031910.c0e0b3bf0872704f385d48808df6a51e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf1/a78695d99c44472fb73a8df5d283aed5.b92c1c7e8ba3ec18a0b4e94867da9855 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf1/a78695d99c44472fb73a8df5d283aed5.b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:56,392 DEBUG [StoreCloser-testSplitWithoutPONR,,1682996031910.c0e0b3bf0872704f385d48808df6a51e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf1/de0cd278f5ae475c8e64f02e18179538.b92c1c7e8ba3ec18a0b4e94867da9855 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf1/de0cd278f5ae475c8e64f02e18179538.b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:56,393 DEBUG [StoreCloser-testSplitWithoutPONR,,1682996031910.c0e0b3bf0872704f385d48808df6a51e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf1/16a5e1fe22774dc782ac22bcb6a70060.b92c1c7e8ba3ec18a0b4e94867da9855 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf1/16a5e1fe22774dc782ac22bcb6a70060.b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:56,394 DEBUG [StoreCloser-testSplitWithoutPONR,,1682996031910.c0e0b3bf0872704f385d48808df6a51e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf1/13ee2b4b0c024358a690edb0b94aac79.b92c1c7e8ba3ec18a0b4e94867da9855 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf1/13ee2b4b0c024358a690edb0b94aac79.b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:56,396 DEBUG [StoreCloser-testSplitWithoutPONR,,1682996031910.c0e0b3bf0872704f385d48808df6a51e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf1/4d8bff8f40d4435e82f15595c5237edd.b92c1c7e8ba3ec18a0b4e94867da9855 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf1/4d8bff8f40d4435e82f15595c5237edd.b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:56,397 DEBUG [StoreCloser-testSplitWithoutPONR,,1682996031910.c0e0b3bf0872704f385d48808df6a51e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf1/8dd2e18ff3e64f1999a3bee1527292e3.b92c1c7e8ba3ec18a0b4e94867da9855 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf1/8dd2e18ff3e64f1999a3bee1527292e3.b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:56,399 DEBUG [StoreCloser-testSplitWithoutPONR,,1682996031910.c0e0b3bf0872704f385d48808df6a51e.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf2/b94fd300967f43f7a6ca454238517875.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/b94fd300967f43f7a6ca454238517875-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf2/8f0768252daf4fca8efafdcc6273b3c6.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/8f0768252daf4fca8efafdcc6273b3c6-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf2/5c6252080bbc48e6b6a023e08c36b716.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/5c6252080bbc48e6b6a023e08c36b716-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf2/488923b0b4ec41659120249ee07156c9.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/488923b0b4ec41659120249ee07156c9-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf2/912fddaa1bc14af3ad9378909904f17f.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/912fddaa1bc14af3ad9378909904f17f-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf2/53ab7c6e85de44f29fcbaf368e39d276.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/53ab7c6e85de44f29fcbaf368e39d276-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf2/427ba387297e4b318fda28a69d090362.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/427ba387297e4b318fda28a69d090362-bottom] to archive 2023-05-02 02:53:56,400 DEBUG [StoreCloser-testSplitWithoutPONR,,1682996031910.c0e0b3bf0872704f385d48808df6a51e.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-02 02:53:56,402 DEBUG [StoreCloser-testSplitWithoutPONR,,1682996031910.c0e0b3bf0872704f385d48808df6a51e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf2/b94fd300967f43f7a6ca454238517875.b92c1c7e8ba3ec18a0b4e94867da9855 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf2/b94fd300967f43f7a6ca454238517875.b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:56,403 DEBUG [StoreCloser-testSplitWithoutPONR,,1682996031910.c0e0b3bf0872704f385d48808df6a51e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf2/8f0768252daf4fca8efafdcc6273b3c6.b92c1c7e8ba3ec18a0b4e94867da9855 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf2/8f0768252daf4fca8efafdcc6273b3c6.b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:56,404 DEBUG [StoreCloser-testSplitWithoutPONR,,1682996031910.c0e0b3bf0872704f385d48808df6a51e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf2/5c6252080bbc48e6b6a023e08c36b716.b92c1c7e8ba3ec18a0b4e94867da9855 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf2/5c6252080bbc48e6b6a023e08c36b716.b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:56,406 DEBUG [StoreCloser-testSplitWithoutPONR,,1682996031910.c0e0b3bf0872704f385d48808df6a51e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf2/488923b0b4ec41659120249ee07156c9.b92c1c7e8ba3ec18a0b4e94867da9855 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf2/488923b0b4ec41659120249ee07156c9.b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:56,407 DEBUG [StoreCloser-testSplitWithoutPONR,,1682996031910.c0e0b3bf0872704f385d48808df6a51e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf2/912fddaa1bc14af3ad9378909904f17f.b92c1c7e8ba3ec18a0b4e94867da9855 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf2/912fddaa1bc14af3ad9378909904f17f.b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:56,408 DEBUG [StoreCloser-testSplitWithoutPONR,,1682996031910.c0e0b3bf0872704f385d48808df6a51e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf2/53ab7c6e85de44f29fcbaf368e39d276.b92c1c7e8ba3ec18a0b4e94867da9855 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf2/53ab7c6e85de44f29fcbaf368e39d276.b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:56,409 DEBUG [StoreCloser-testSplitWithoutPONR,,1682996031910.c0e0b3bf0872704f385d48808df6a51e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf2/427ba387297e4b318fda28a69d090362.b92c1c7e8ba3ec18a0b4e94867da9855 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf2/427ba387297e4b318fda28a69d090362.b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:56,413 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-05-02 02:53:56,413 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-02 02:53:56,414 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitWithoutPONR,,1682996031910.c0e0b3bf0872704f385d48808df6a51e. 2023-05-02 02:53:56,414 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for c0e0b3bf0872704f385d48808df6a51e: 2023-05-02 02:53:56,415 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed c0e0b3bf0872704f385d48808df6a51e 2023-05-02 02:53:56,415 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:56,415 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 6e3e738dce7ec3e6ddbdadab93b4596e, disabling compactions & flushes 2023-05-02 02:53:56,416 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitWithoutPONR,41,1682996031910.6e3e738dce7ec3e6ddbdadab93b4596e. 2023-05-02 02:53:56,416 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitWithoutPONR,41,1682996031910.6e3e738dce7ec3e6ddbdadab93b4596e. 2023-05-02 02:53:56,416 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitWithoutPONR,41,1682996031910.6e3e738dce7ec3e6ddbdadab93b4596e. after waiting 0 ms 2023-05-02 02:53:56,416 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitWithoutPONR,41,1682996031910.6e3e738dce7ec3e6ddbdadab93b4596e. 2023-05-02 02:53:56,416 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=c0e0b3bf0872704f385d48808df6a51e, regionState=CLOSED 2023-05-02 02:53:56,416 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1682996031910.c0e0b3bf0872704f385d48808df6a51e.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1682996036416"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682996036416"}]},"ts":"1682996036416"} 2023-05-02 02:53:56,417 DEBUG [StoreCloser-testSplitWithoutPONR,41,1682996031910.6e3e738dce7ec3e6ddbdadab93b4596e.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf1/5317f0b5b225487c82a9f79b071b429d.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/5317f0b5b225487c82a9f79b071b429d-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf1/a78695d99c44472fb73a8df5d283aed5.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/a78695d99c44472fb73a8df5d283aed5-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf1/de0cd278f5ae475c8e64f02e18179538.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/de0cd278f5ae475c8e64f02e18179538-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf1/16a5e1fe22774dc782ac22bcb6a70060.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/16a5e1fe22774dc782ac22bcb6a70060-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf1/13ee2b4b0c024358a690edb0b94aac79.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/13ee2b4b0c024358a690edb0b94aac79-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf1/4d8bff8f40d4435e82f15595c5237edd.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/4d8bff8f40d4435e82f15595c5237edd-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf1/8dd2e18ff3e64f1999a3bee1527292e3.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/8dd2e18ff3e64f1999a3bee1527292e3-top] to archive 2023-05-02 02:53:56,418 DEBUG [StoreCloser-testSplitWithoutPONR,41,1682996031910.6e3e738dce7ec3e6ddbdadab93b4596e.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-02 02:53:56,420 DEBUG [StoreCloser-testSplitWithoutPONR,41,1682996031910.6e3e738dce7ec3e6ddbdadab93b4596e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf1/5317f0b5b225487c82a9f79b071b429d.b92c1c7e8ba3ec18a0b4e94867da9855 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf1/5317f0b5b225487c82a9f79b071b429d.b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:56,420 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=39, resume processing ppid=36 2023-05-02 02:53:56,420 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=39, ppid=36, state=SUCCESS; CloseRegionProcedure c0e0b3bf0872704f385d48808df6a51e, server=jenkins-hbase4.apache.org,42019,1682996021676 in 192 msec 2023-05-02 02:53:56,421 DEBUG [StoreCloser-testSplitWithoutPONR,41,1682996031910.6e3e738dce7ec3e6ddbdadab93b4596e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf1/a78695d99c44472fb73a8df5d283aed5.b92c1c7e8ba3ec18a0b4e94867da9855 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf1/a78695d99c44472fb73a8df5d283aed5.b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:56,422 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=36, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=c0e0b3bf0872704f385d48808df6a51e, UNASSIGN in 202 msec 2023-05-02 02:53:56,423 DEBUG [StoreCloser-testSplitWithoutPONR,41,1682996031910.6e3e738dce7ec3e6ddbdadab93b4596e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf1/de0cd278f5ae475c8e64f02e18179538.b92c1c7e8ba3ec18a0b4e94867da9855 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf1/de0cd278f5ae475c8e64f02e18179538.b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:56,424 DEBUG [StoreCloser-testSplitWithoutPONR,41,1682996031910.6e3e738dce7ec3e6ddbdadab93b4596e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf1/16a5e1fe22774dc782ac22bcb6a70060.b92c1c7e8ba3ec18a0b4e94867da9855 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf1/16a5e1fe22774dc782ac22bcb6a70060.b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:56,425 DEBUG [StoreCloser-testSplitWithoutPONR,41,1682996031910.6e3e738dce7ec3e6ddbdadab93b4596e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf1/13ee2b4b0c024358a690edb0b94aac79.b92c1c7e8ba3ec18a0b4e94867da9855 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf1/13ee2b4b0c024358a690edb0b94aac79.b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:56,426 DEBUG [StoreCloser-testSplitWithoutPONR,41,1682996031910.6e3e738dce7ec3e6ddbdadab93b4596e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf1/4d8bff8f40d4435e82f15595c5237edd.b92c1c7e8ba3ec18a0b4e94867da9855 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf1/4d8bff8f40d4435e82f15595c5237edd.b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:56,427 DEBUG [StoreCloser-testSplitWithoutPONR,41,1682996031910.6e3e738dce7ec3e6ddbdadab93b4596e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf1/8dd2e18ff3e64f1999a3bee1527292e3.b92c1c7e8ba3ec18a0b4e94867da9855 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf1/8dd2e18ff3e64f1999a3bee1527292e3.b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:56,429 DEBUG [StoreCloser-testSplitWithoutPONR,41,1682996031910.6e3e738dce7ec3e6ddbdadab93b4596e.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf2/b94fd300967f43f7a6ca454238517875.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/b94fd300967f43f7a6ca454238517875-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf2/8f0768252daf4fca8efafdcc6273b3c6.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/8f0768252daf4fca8efafdcc6273b3c6-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf2/5c6252080bbc48e6b6a023e08c36b716.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/5c6252080bbc48e6b6a023e08c36b716-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf2/488923b0b4ec41659120249ee07156c9.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/488923b0b4ec41659120249ee07156c9-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf2/912fddaa1bc14af3ad9378909904f17f.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/912fddaa1bc14af3ad9378909904f17f-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf2/53ab7c6e85de44f29fcbaf368e39d276.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/53ab7c6e85de44f29fcbaf368e39d276-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf2/427ba387297e4b318fda28a69d090362.b92c1c7e8ba3ec18a0b4e94867da9855->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/427ba387297e4b318fda28a69d090362-top] to archive 2023-05-02 02:53:56,430 DEBUG [StoreCloser-testSplitWithoutPONR,41,1682996031910.6e3e738dce7ec3e6ddbdadab93b4596e.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-02 02:53:56,431 DEBUG [StoreCloser-testSplitWithoutPONR,41,1682996031910.6e3e738dce7ec3e6ddbdadab93b4596e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf2/b94fd300967f43f7a6ca454238517875.b92c1c7e8ba3ec18a0b4e94867da9855 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf2/b94fd300967f43f7a6ca454238517875.b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:56,432 DEBUG [StoreCloser-testSplitWithoutPONR,41,1682996031910.6e3e738dce7ec3e6ddbdadab93b4596e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf2/8f0768252daf4fca8efafdcc6273b3c6.b92c1c7e8ba3ec18a0b4e94867da9855 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf2/8f0768252daf4fca8efafdcc6273b3c6.b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:56,433 DEBUG [StoreCloser-testSplitWithoutPONR,41,1682996031910.6e3e738dce7ec3e6ddbdadab93b4596e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf2/5c6252080bbc48e6b6a023e08c36b716.b92c1c7e8ba3ec18a0b4e94867da9855 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf2/5c6252080bbc48e6b6a023e08c36b716.b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:56,434 DEBUG [StoreCloser-testSplitWithoutPONR,41,1682996031910.6e3e738dce7ec3e6ddbdadab93b4596e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf2/488923b0b4ec41659120249ee07156c9.b92c1c7e8ba3ec18a0b4e94867da9855 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf2/488923b0b4ec41659120249ee07156c9.b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:56,435 DEBUG [StoreCloser-testSplitWithoutPONR,41,1682996031910.6e3e738dce7ec3e6ddbdadab93b4596e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf2/912fddaa1bc14af3ad9378909904f17f.b92c1c7e8ba3ec18a0b4e94867da9855 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf2/912fddaa1bc14af3ad9378909904f17f.b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:56,437 DEBUG [StoreCloser-testSplitWithoutPONR,41,1682996031910.6e3e738dce7ec3e6ddbdadab93b4596e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf2/53ab7c6e85de44f29fcbaf368e39d276.b92c1c7e8ba3ec18a0b4e94867da9855 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf2/53ab7c6e85de44f29fcbaf368e39d276.b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:56,438 DEBUG [StoreCloser-testSplitWithoutPONR,41,1682996031910.6e3e738dce7ec3e6ddbdadab93b4596e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf2/427ba387297e4b318fda28a69d090362.b92c1c7e8ba3ec18a0b4e94867da9855 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf2/427ba387297e4b318fda28a69d090362.b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:56,442 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-05-02 02:53:56,442 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-02 02:53:56,442 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitWithoutPONR,41,1682996031910.6e3e738dce7ec3e6ddbdadab93b4596e. 2023-05-02 02:53:56,442 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 6e3e738dce7ec3e6ddbdadab93b4596e: 2023-05-02 02:53:56,444 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:56,445 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=6e3e738dce7ec3e6ddbdadab93b4596e, regionState=CLOSED 2023-05-02 02:53:56,445 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,41,1682996031910.6e3e738dce7ec3e6ddbdadab93b4596e.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1682996036445"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682996036445"}]},"ts":"1682996036445"} 2023-05-02 02:53:56,449 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=38, resume processing ppid=37 2023-05-02 02:53:56,449 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=38, ppid=37, state=SUCCESS; CloseRegionProcedure 6e3e738dce7ec3e6ddbdadab93b4596e, server=jenkins-hbase4.apache.org,42019,1682996021676 in 225 msec 2023-05-02 02:53:56,451 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=37, resume processing ppid=35 2023-05-02 02:53:56,451 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=37, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6e3e738dce7ec3e6ddbdadab93b4596e, UNASSIGN in 231 msec 2023-05-02 02:53:56,452 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682996036452"}]},"ts":"1682996036452"} 2023-05-02 02:53:56,454 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLED in hbase:meta 2023-05-02 02:53:56,456 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitWithoutPONR to state=DISABLED 2023-05-02 02:53:56,458 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR in 247 msec 2023-05-02 02:53:56,516 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-05-02 02:53:56,516 INFO [Listener at localhost/37119] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitWithoutPONR, procId: 35 completed 2023-05-02 02:53:56,517 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testSplitWithoutPONR 2023-05-02 02:53:56,518 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure2.ProcedureExecutor(1029): Stored pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-02 02:53:56,520 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-02 02:53:56,521 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=40, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-02 02:53:56,521 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1227): Checking to see if procedure is done pid=40 2023-05-02 02:53:56,529 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:56,529 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e 2023-05-02 02:53:56,529 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:56,531 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1, FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2, FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/recovered.edits] 2023-05-02 02:53:56,532 DEBUG [HFileArchiver-1] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/6e3e738dce7ec3e6ddbdadab93b4596e, FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf1, FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf2, FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/recovered.edits] 2023-05-02 02:53:56,532 DEBUG [HFileArchiver-8] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/c0e0b3bf0872704f385d48808df6a51e, FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf1, FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf2, FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/recovered.edits] 2023-05-02 02:53:56,536 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/6e3e738dce7ec3e6ddbdadab93b4596e/.regioninfo to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/6e3e738dce7ec3e6ddbdadab93b4596e/.regioninfo 2023-05-02 02:53:56,536 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/c0e0b3bf0872704f385d48808df6a51e/.regioninfo to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/c0e0b3bf0872704f385d48808df6a51e/.regioninfo 2023-05-02 02:53:56,536 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/13ee2b4b0c024358a690edb0b94aac79 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/13ee2b4b0c024358a690edb0b94aac79 2023-05-02 02:53:56,539 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/16a5e1fe22774dc782ac22bcb6a70060 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/16a5e1fe22774dc782ac22bcb6a70060 2023-05-02 02:53:56,540 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/6e3e738dce7ec3e6ddbdadab93b4596e/cf1/13ee2b4b0c024358a690edb0b94aac79.b92c1c7e8ba3ec18a0b4e94867da9855 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/6e3e738dce7ec3e6ddbdadab93b4596e/cf1/13ee2b4b0c024358a690edb0b94aac79.b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:56,540 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/c0e0b3bf0872704f385d48808df6a51e/cf1/13ee2b4b0c024358a690edb0b94aac79.b92c1c7e8ba3ec18a0b4e94867da9855 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/c0e0b3bf0872704f385d48808df6a51e/cf1/13ee2b4b0c024358a690edb0b94aac79.b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:56,540 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/4d8bff8f40d4435e82f15595c5237edd to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/4d8bff8f40d4435e82f15595c5237edd 2023-05-02 02:53:56,541 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/6e3e738dce7ec3e6ddbdadab93b4596e/cf1/16a5e1fe22774dc782ac22bcb6a70060.b92c1c7e8ba3ec18a0b4e94867da9855 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/6e3e738dce7ec3e6ddbdadab93b4596e/cf1/16a5e1fe22774dc782ac22bcb6a70060.b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:56,542 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/c0e0b3bf0872704f385d48808df6a51e/cf1/16a5e1fe22774dc782ac22bcb6a70060.b92c1c7e8ba3ec18a0b4e94867da9855 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/c0e0b3bf0872704f385d48808df6a51e/cf1/16a5e1fe22774dc782ac22bcb6a70060.b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:56,542 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/5317f0b5b225487c82a9f79b071b429d to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/5317f0b5b225487c82a9f79b071b429d 2023-05-02 02:53:56,543 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/6e3e738dce7ec3e6ddbdadab93b4596e/cf1/4d8bff8f40d4435e82f15595c5237edd.b92c1c7e8ba3ec18a0b4e94867da9855 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/6e3e738dce7ec3e6ddbdadab93b4596e/cf1/4d8bff8f40d4435e82f15595c5237edd.b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:56,543 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/c0e0b3bf0872704f385d48808df6a51e/cf1/4d8bff8f40d4435e82f15595c5237edd.b92c1c7e8ba3ec18a0b4e94867da9855 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/c0e0b3bf0872704f385d48808df6a51e/cf1/4d8bff8f40d4435e82f15595c5237edd.b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:56,543 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/8dd2e18ff3e64f1999a3bee1527292e3 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/8dd2e18ff3e64f1999a3bee1527292e3 2023-05-02 02:53:56,545 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/c0e0b3bf0872704f385d48808df6a51e/cf1/5317f0b5b225487c82a9f79b071b429d.b92c1c7e8ba3ec18a0b4e94867da9855 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/c0e0b3bf0872704f385d48808df6a51e/cf1/5317f0b5b225487c82a9f79b071b429d.b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:56,545 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/6e3e738dce7ec3e6ddbdadab93b4596e/cf1/5317f0b5b225487c82a9f79b071b429d.b92c1c7e8ba3ec18a0b4e94867da9855 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/6e3e738dce7ec3e6ddbdadab93b4596e/cf1/5317f0b5b225487c82a9f79b071b429d.b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:56,545 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/a78695d99c44472fb73a8df5d283aed5 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/a78695d99c44472fb73a8df5d283aed5 2023-05-02 02:53:56,546 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/c0e0b3bf0872704f385d48808df6a51e/cf1/8dd2e18ff3e64f1999a3bee1527292e3.b92c1c7e8ba3ec18a0b4e94867da9855 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/c0e0b3bf0872704f385d48808df6a51e/cf1/8dd2e18ff3e64f1999a3bee1527292e3.b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:56,546 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/6e3e738dce7ec3e6ddbdadab93b4596e/cf1/8dd2e18ff3e64f1999a3bee1527292e3.b92c1c7e8ba3ec18a0b4e94867da9855 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/6e3e738dce7ec3e6ddbdadab93b4596e/cf1/8dd2e18ff3e64f1999a3bee1527292e3.b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:56,547 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/de0cd278f5ae475c8e64f02e18179538 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf1/de0cd278f5ae475c8e64f02e18179538 2023-05-02 02:53:56,548 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/c0e0b3bf0872704f385d48808df6a51e/cf1/a78695d99c44472fb73a8df5d283aed5.b92c1c7e8ba3ec18a0b4e94867da9855 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/c0e0b3bf0872704f385d48808df6a51e/cf1/a78695d99c44472fb73a8df5d283aed5.b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:56,548 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/6e3e738dce7ec3e6ddbdadab93b4596e/cf1/a78695d99c44472fb73a8df5d283aed5.b92c1c7e8ba3ec18a0b4e94867da9855 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/6e3e738dce7ec3e6ddbdadab93b4596e/cf1/a78695d99c44472fb73a8df5d283aed5.b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:56,550 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/c0e0b3bf0872704f385d48808df6a51e/cf1/de0cd278f5ae475c8e64f02e18179538.b92c1c7e8ba3ec18a0b4e94867da9855 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/c0e0b3bf0872704f385d48808df6a51e/cf1/de0cd278f5ae475c8e64f02e18179538.b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:56,550 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/6e3e738dce7ec3e6ddbdadab93b4596e/cf1/de0cd278f5ae475c8e64f02e18179538.b92c1c7e8ba3ec18a0b4e94867da9855 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/6e3e738dce7ec3e6ddbdadab93b4596e/cf1/de0cd278f5ae475c8e64f02e18179538.b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:56,551 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/427ba387297e4b318fda28a69d090362 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/427ba387297e4b318fda28a69d090362 2023-05-02 02:53:56,553 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/488923b0b4ec41659120249ee07156c9 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/488923b0b4ec41659120249ee07156c9 2023-05-02 02:53:56,555 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/6e3e738dce7ec3e6ddbdadab93b4596e/cf2/427ba387297e4b318fda28a69d090362.b92c1c7e8ba3ec18a0b4e94867da9855 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/6e3e738dce7ec3e6ddbdadab93b4596e/cf2/427ba387297e4b318fda28a69d090362.b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:56,555 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/c0e0b3bf0872704f385d48808df6a51e/cf2/427ba387297e4b318fda28a69d090362.b92c1c7e8ba3ec18a0b4e94867da9855 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/c0e0b3bf0872704f385d48808df6a51e/cf2/427ba387297e4b318fda28a69d090362.b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:56,556 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/53ab7c6e85de44f29fcbaf368e39d276 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/53ab7c6e85de44f29fcbaf368e39d276 2023-05-02 02:53:56,557 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/6e3e738dce7ec3e6ddbdadab93b4596e/cf2/488923b0b4ec41659120249ee07156c9.b92c1c7e8ba3ec18a0b4e94867da9855 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/6e3e738dce7ec3e6ddbdadab93b4596e/cf2/488923b0b4ec41659120249ee07156c9.b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:56,557 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/c0e0b3bf0872704f385d48808df6a51e/cf2/488923b0b4ec41659120249ee07156c9.b92c1c7e8ba3ec18a0b4e94867da9855 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/c0e0b3bf0872704f385d48808df6a51e/cf2/488923b0b4ec41659120249ee07156c9.b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:56,559 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/5c6252080bbc48e6b6a023e08c36b716 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/5c6252080bbc48e6b6a023e08c36b716 2023-05-02 02:53:56,559 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/6e3e738dce7ec3e6ddbdadab93b4596e/cf2/53ab7c6e85de44f29fcbaf368e39d276.b92c1c7e8ba3ec18a0b4e94867da9855 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/6e3e738dce7ec3e6ddbdadab93b4596e/cf2/53ab7c6e85de44f29fcbaf368e39d276.b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:56,560 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/c0e0b3bf0872704f385d48808df6a51e/cf2/53ab7c6e85de44f29fcbaf368e39d276.b92c1c7e8ba3ec18a0b4e94867da9855 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/c0e0b3bf0872704f385d48808df6a51e/cf2/53ab7c6e85de44f29fcbaf368e39d276.b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:56,565 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/8f0768252daf4fca8efafdcc6273b3c6 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/8f0768252daf4fca8efafdcc6273b3c6 2023-05-02 02:53:56,565 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/6e3e738dce7ec3e6ddbdadab93b4596e/cf2/5c6252080bbc48e6b6a023e08c36b716.b92c1c7e8ba3ec18a0b4e94867da9855 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/6e3e738dce7ec3e6ddbdadab93b4596e/cf2/5c6252080bbc48e6b6a023e08c36b716.b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:56,566 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/c0e0b3bf0872704f385d48808df6a51e/cf2/5c6252080bbc48e6b6a023e08c36b716.b92c1c7e8ba3ec18a0b4e94867da9855 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/c0e0b3bf0872704f385d48808df6a51e/cf2/5c6252080bbc48e6b6a023e08c36b716.b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:56,567 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/912fddaa1bc14af3ad9378909904f17f to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/912fddaa1bc14af3ad9378909904f17f 2023-05-02 02:53:56,567 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/6e3e738dce7ec3e6ddbdadab93b4596e/cf2/8f0768252daf4fca8efafdcc6273b3c6.b92c1c7e8ba3ec18a0b4e94867da9855 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/6e3e738dce7ec3e6ddbdadab93b4596e/cf2/8f0768252daf4fca8efafdcc6273b3c6.b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:56,568 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/c0e0b3bf0872704f385d48808df6a51e/cf2/8f0768252daf4fca8efafdcc6273b3c6.b92c1c7e8ba3ec18a0b4e94867da9855 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/c0e0b3bf0872704f385d48808df6a51e/cf2/8f0768252daf4fca8efafdcc6273b3c6.b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:56,569 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/b94fd300967f43f7a6ca454238517875 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/cf2/b94fd300967f43f7a6ca454238517875 2023-05-02 02:53:56,570 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/6e3e738dce7ec3e6ddbdadab93b4596e/cf2/912fddaa1bc14af3ad9378909904f17f.b92c1c7e8ba3ec18a0b4e94867da9855 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/6e3e738dce7ec3e6ddbdadab93b4596e/cf2/912fddaa1bc14af3ad9378909904f17f.b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:56,570 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/c0e0b3bf0872704f385d48808df6a51e/cf2/912fddaa1bc14af3ad9378909904f17f.b92c1c7e8ba3ec18a0b4e94867da9855 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/c0e0b3bf0872704f385d48808df6a51e/cf2/912fddaa1bc14af3ad9378909904f17f.b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:56,572 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/c0e0b3bf0872704f385d48808df6a51e/cf2/b94fd300967f43f7a6ca454238517875.b92c1c7e8ba3ec18a0b4e94867da9855 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/c0e0b3bf0872704f385d48808df6a51e/cf2/b94fd300967f43f7a6ca454238517875.b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:56,573 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/6e3e738dce7ec3e6ddbdadab93b4596e/cf2/b94fd300967f43f7a6ca454238517875.b92c1c7e8ba3ec18a0b4e94867da9855 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/6e3e738dce7ec3e6ddbdadab93b4596e/cf2/b94fd300967f43f7a6ca454238517875.b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:56,574 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/recovered.edits/85.seqid to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855/recovered.edits/85.seqid 2023-05-02 02:53:56,574 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/b92c1c7e8ba3ec18a0b4e94867da9855 2023-05-02 02:53:56,575 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf1/a9b141707bf14943b1f5a27a40c906c0 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf1/a9b141707bf14943b1f5a27a40c906c0 2023-05-02 02:53:56,576 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf1/ce982974d1bc4e85b6890d1cdeffeb2b to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf1/ce982974d1bc4e85b6890d1cdeffeb2b 2023-05-02 02:53:56,578 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf2/e9d8bbfb5ab04c19a12d8424d2d6c45c to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/cf2/e9d8bbfb5ab04c19a12d8424d2d6c45c 2023-05-02 02:53:56,579 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf2/806fb6b1551f4e418ab2610151e25f35 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/cf2/806fb6b1551f4e418ab2610151e25f35 2023-05-02 02:53:56,582 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/recovered.edits/90.seqid to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e/recovered.edits/90.seqid 2023-05-02 02:53:56,582 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/recovered.edits/90.seqid to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e/recovered.edits/90.seqid 2023-05-02 02:53:56,583 DEBUG [HFileArchiver-8] backup.HFileArchiver(596): Deleted hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/c0e0b3bf0872704f385d48808df6a51e 2023-05-02 02:53:56,583 DEBUG [HFileArchiver-1] backup.HFileArchiver(596): Deleted hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitWithoutPONR/6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:56,583 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-05-02 02:53:56,586 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=40, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-02 02:53:56,589 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitWithoutPONR from hbase:meta 2023-05-02 02:53:56,591 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitWithoutPONR' descriptor. 2023-05-02 02:53:56,593 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=40, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-02 02:53:56,593 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitWithoutPONR' from region states. 2023-05-02 02:53:56,593 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1682996029879.b92c1c7e8ba3ec18a0b4e94867da9855.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1682996036593"}]},"ts":"9223372036854775807"} 2023-05-02 02:53:56,593 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1682996031910.c0e0b3bf0872704f385d48808df6a51e.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1682996036593"}]},"ts":"9223372036854775807"} 2023-05-02 02:53:56,593 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,41,1682996031910.6e3e738dce7ec3e6ddbdadab93b4596e.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1682996036593"}]},"ts":"9223372036854775807"} 2023-05-02 02:53:56,596 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-05-02 02:53:56,596 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => b92c1c7e8ba3ec18a0b4e94867da9855, NAME => 'testSplitWithoutPONR,,1682996029879.b92c1c7e8ba3ec18a0b4e94867da9855.', STARTKEY => '', ENDKEY => ''}, {ENCODED => c0e0b3bf0872704f385d48808df6a51e, NAME => 'testSplitWithoutPONR,,1682996031910.c0e0b3bf0872704f385d48808df6a51e.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 6e3e738dce7ec3e6ddbdadab93b4596e, NAME => 'testSplitWithoutPONR,41,1682996031910.6e3e738dce7ec3e6ddbdadab93b4596e.', STARTKEY => '41', ENDKEY => ''}] 2023-05-02 02:53:56,596 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitWithoutPONR' as deleted. 2023-05-02 02:53:56,596 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1682996036596"}]},"ts":"9223372036854775807"} 2023-05-02 02:53:56,598 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitWithoutPONR state from META 2023-05-02 02:53:56,600 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=40, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-02 02:53:56,602 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR in 83 msec 2023-05-02 02:53:56,623 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1227): Checking to see if procedure is done pid=40 2023-05-02 02:53:56,623 INFO [Listener at localhost/37119] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitWithoutPONR, procId: 40 completed 2023-05-02 02:53:56,639 INFO [Listener at localhost/37119] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=399 (was 390) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_912135922_15 at /127.0.0.1:47740 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_912135922_15 at /127.0.0.1:47758 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_912135922_15 at /127.0.0.1:47798 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_912135922_15 at /127.0.0.1:47774 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_912135922_15 at /127.0.0.1:34200 [Waiting for operation #35] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:2;jenkins-hbase4:42019-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_912135922_15 at /127.0.0.1:45076 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x1f586e56-shared-pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e48d2209-986f-dd5e-f2e4-019027d7bcac/cluster_9b21c39f-0840-34d4-ae73-45b092a6b580/dfs/data/data2/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x1f586e56-shared-pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2c67b29-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e48d2209-986f-dd5e-f2e4-019027d7bcac/cluster_9b21c39f-0840-34d4-ae73-45b092a6b580/dfs/data/data6/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2c67b29-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_912135922_15 at /127.0.0.1:38512 [Waiting for operation #26] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_912135922_15 at /127.0.0.1:47788 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_912135922_15 at /127.0.0.1:45092 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e48d2209-986f-dd5e-f2e4-019027d7bcac/cluster_9b21c39f-0840-34d4-ae73-45b092a6b580/dfs/data/data5/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2c67b29-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_912135922_15 at /127.0.0.1:45042 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2c67b29-metaLookup-shared--pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e48d2209-986f-dd5e-f2e4-019027d7bcac/cluster_9b21c39f-0840-34d4-ae73-45b092a6b580/dfs/data/data1/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_912135922_15 at /127.0.0.1:33742 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2c67b29-metaLookup-shared--pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_912135922_15 at /127.0.0.1:33756 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_912135922_15 at /127.0.0.1:47780 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_912135922_15 at /127.0.0.1:47830 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_912135922_15 at /127.0.0.1:47778 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2c67b29-shared-pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_912135922_15 at /127.0.0.1:38584 [Waiting for operation #36] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=848 (was 842) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=289 (was 306), ProcessCount=170 (was 170), AvailableMemoryMB=13967 (was 13974) 2023-05-02 02:53:56,656 INFO [Listener at localhost/37119] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=399, OpenFileDescriptor=848, MaxFileDescriptor=60000, SystemLoadAverage=289, ProcessCount=170, AvailableMemoryMB=13966 2023-05-02 02:53:56,656 WARN [Listener at localhost/37119] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-02 02:53:56,664 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-05-02 02:53:56,665 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-05-02 02:53:56,667 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(1029): Stored pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-02 02:53:56,669 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-05-02 02:53:56,669 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-05-02 02:53:56,671 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:53:56,672 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a empty. 2023-05-02 02:53:56,672 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:53:56,672 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-05-02 02:53:56,687 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRollbackAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-05-02 02:53:56,688 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(7675): creating {ENCODED => 42fc9cc7beccd63fc2605b5f0fb93a8a, NAME => 'testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp 2023-05-02 02:53:56,699 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(866): Instantiated testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-02 02:53:56,700 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1604): Closing 42fc9cc7beccd63fc2605b5f0fb93a8a, disabling compactions & flushes 2023-05-02 02:53:56,700 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1626): Closing region testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a. 2023-05-02 02:53:56,700 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a. 2023-05-02 02:53:56,700 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1714): Acquired close lock on testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a. after waiting 0 ms 2023-05-02 02:53:56,700 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1724): Updates disabled for region testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a. 2023-05-02 02:53:56,700 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1838): Closed testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a. 2023-05-02 02:53:56,700 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1558): Region close journal for 42fc9cc7beccd63fc2605b5f0fb93a8a: 2023-05-02 02:53:56,703 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-05-02 02:53:56,704 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1682996036703"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682996036703"}]},"ts":"1682996036703"} 2023-05-02 02:53:56,705 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-05-02 02:53:56,706 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-05-02 02:53:56,706 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682996036706"}]},"ts":"1682996036706"} 2023-05-02 02:53:56,707 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLING in hbase:meta 2023-05-02 02:53:56,710 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-05-02 02:53:56,711 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-05-02 02:53:56,711 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-05-02 02:53:56,711 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-05-02 02:53:56,711 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-05-02 02:53:56,711 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, ASSIGN}] 2023-05-02 02:53:56,713 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, ASSIGN 2023-05-02 02:53:56,714 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,39081,1682996021579; forceNewPlan=false, retain=false 2023-05-02 02:53:56,864 INFO [jenkins-hbase4:41119] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-05-02 02:53:56,865 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=42fc9cc7beccd63fc2605b5f0fb93a8a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:56,865 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1682996036865"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1682996036865"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682996036865"}]},"ts":"1682996036865"} 2023-05-02 02:53:56,867 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=43, ppid=42, state=RUNNABLE; OpenRegionProcedure 42fc9cc7beccd63fc2605b5f0fb93a8a, server=jenkins-hbase4.apache.org,39081,1682996021579}] 2023-05-02 02:53:57,020 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,020 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-05-02 02:53:57,024 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52184, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-05-02 02:53:57,030 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a. 2023-05-02 02:53:57,030 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 42fc9cc7beccd63fc2605b5f0fb93a8a, NAME => 'testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a.', STARTKEY => '', ENDKEY => ''} 2023-05-02 02:53:57,031 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-02 02:53:57,031 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:53:57,031 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-02 02:53:57,031 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:53:57,031 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:53:57,032 INFO [StoreOpener-42fc9cc7beccd63fc2605b5f0fb93a8a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:53:57,034 DEBUG [StoreOpener-42fc9cc7beccd63fc2605b5f0fb93a8a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1 2023-05-02 02:53:57,034 DEBUG [StoreOpener-42fc9cc7beccd63fc2605b5f0fb93a8a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1 2023-05-02 02:53:57,034 INFO [StoreOpener-42fc9cc7beccd63fc2605b5f0fb93a8a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 42fc9cc7beccd63fc2605b5f0fb93a8a columnFamilyName cf1 2023-05-02 02:53:57,035 INFO [StoreOpener-42fc9cc7beccd63fc2605b5f0fb93a8a-1] regionserver.HStore(310): Store=42fc9cc7beccd63fc2605b5f0fb93a8a/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-02 02:53:57,035 INFO [StoreOpener-42fc9cc7beccd63fc2605b5f0fb93a8a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:53:57,036 DEBUG [StoreOpener-42fc9cc7beccd63fc2605b5f0fb93a8a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2 2023-05-02 02:53:57,036 DEBUG [StoreOpener-42fc9cc7beccd63fc2605b5f0fb93a8a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2 2023-05-02 02:53:57,037 INFO [StoreOpener-42fc9cc7beccd63fc2605b5f0fb93a8a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 42fc9cc7beccd63fc2605b5f0fb93a8a columnFamilyName cf2 2023-05-02 02:53:57,037 INFO [StoreOpener-42fc9cc7beccd63fc2605b5f0fb93a8a-1] regionserver.HStore(310): Store=42fc9cc7beccd63fc2605b5f0fb93a8a/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-02 02:53:57,038 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:53:57,038 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:53:57,040 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-02 02:53:57,041 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:53:57,044 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-05-02 02:53:57,045 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 42fc9cc7beccd63fc2605b5f0fb93a8a; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72674626, jitterRate=0.08293631672859192}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-02 02:53:57,045 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 42fc9cc7beccd63fc2605b5f0fb93a8a: 2023-05-02 02:53:57,046 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a., pid=43, masterSystemTime=1682996037020 2023-05-02 02:53:57,049 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a. 2023-05-02 02:53:57,049 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a. 2023-05-02 02:53:57,050 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=42fc9cc7beccd63fc2605b5f0fb93a8a, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,050 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1682996037050"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1682996037050"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1682996037050"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682996037050"}]},"ts":"1682996037050"} 2023-05-02 02:53:57,053 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=43, resume processing ppid=42 2023-05-02 02:53:57,053 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=43, ppid=42, state=SUCCESS; OpenRegionProcedure 42fc9cc7beccd63fc2605b5f0fb93a8a, server=jenkins-hbase4.apache.org,39081,1682996021579 in 184 msec 2023-05-02 02:53:57,056 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=42, resume processing ppid=41 2023-05-02 02:53:57,056 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=42, ppid=41, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, ASSIGN in 343 msec 2023-05-02 02:53:57,057 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-05-02 02:53:57,057 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682996037057"}]},"ts":"1682996037057"} 2023-05-02 02:53:57,059 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLED in hbase:meta 2023-05-02 02:53:57,066 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-05-02 02:53:57,068 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution in 401 msec 2023-05-02 02:53:57,169 DEBUG [Listener at localhost/37119] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-05-02 02:53:57,174 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52194, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-05-02 02:53:57,181 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-02 02:53:57,183 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-05-02 02:53:57,183 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-05-02 02:53:57,183 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:57,183 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-05-02 02:53:57,183 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-05-02 02:53:57,184 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,184 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-02 02:53:57,185 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:57,185 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:57,185 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:57,185 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:57,185 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,185 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:57,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:57,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:57,186 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,186 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:57,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:57,186 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-02 02:53:57,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-02 02:53:57,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-02 02:53:57,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-02 02:53:57,187 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,187 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,187 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:53:57,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-02 02:53:57,187 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-02 02:53:57,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:53:57,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-02 02:53:57,188 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-02 02:53:57,188 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-02 02:53:57,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:53:57,188 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:57,188 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-02 02:53:57,188 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-02 02:53:57,188 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-02 02:53:57,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-02 02:53:57,188 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42019,1682996021676' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-02 02:53:57,188 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-02 02:53:57,188 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-02 02:53:57,189 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-02 02:53:57,189 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:57,189 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-02 02:53:57,189 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-02 02:53:57,189 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38925,1682996021633' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-02 02:53:57,189 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-02 02:53:57,189 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-02 02:53:57,189 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:57,189 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-02 02:53:57,190 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,191 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,191 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,191 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:53:57,191 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,191 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:53:57,192 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-02 02:53:57,192 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-02 02:53:57,192 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a. 2023-05-02 02:53:57,192 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a. started... 2023-05-02 02:53:57,193 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 42fc9cc7beccd63fc2605b5f0fb93a8a 2/2 column families, dataSize=120 B heapSize=928 B 2023-05-02 02:53:57,220 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/.tmp/cf1/bde0a9d3ebfa4e1996e42b844971f197 2023-05-02 02:53:57,237 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/.tmp/cf2/65211868ead84a0895c2ebf9105167e0 2023-05-02 02:53:57,242 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/.tmp/cf1/bde0a9d3ebfa4e1996e42b844971f197 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/bde0a9d3ebfa4e1996e42b844971f197 2023-05-02 02:53:57,247 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/bde0a9d3ebfa4e1996e42b844971f197, entries=2, sequenceid=6, filesize=4.8 K 2023-05-02 02:53:57,248 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/.tmp/cf2/65211868ead84a0895c2ebf9105167e0 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/65211868ead84a0895c2ebf9105167e0 2023-05-02 02:53:57,253 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/65211868ead84a0895c2ebf9105167e0, entries=2, sequenceid=6, filesize=4.8 K 2023-05-02 02:53:57,255 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 42fc9cc7beccd63fc2605b5f0fb93a8a in 61ms, sequenceid=6, compaction requested=false 2023-05-02 02:53:57,255 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackAndDoubleExecution' 2023-05-02 02:53:57,256 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 42fc9cc7beccd63fc2605b5f0fb93a8a: 2023-05-02 02:53:57,256 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a. 2023-05-02 02:53:57,256 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-02 02:53:57,256 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-02 02:53:57,256 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,256 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-02 02:53:57,256 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-02 02:53:57,259 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,259 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:53:57,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:53:57,260 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,260 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:53:57,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:53:57,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:53:57,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-02 02:53:57,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:57,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:57,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:53:57,263 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-05-02 02:53:57,263 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6c66f9f4[Count = 0] remaining members to acquire global barrier 2023-05-02 02:53:57,264 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-05-02 02:53:57,264 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,265 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,265 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,265 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,265 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,265 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,265 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-02 02:53:57,265 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-02 02:53:57,265 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,265 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-02 02:53:57,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,265 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,265 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-02 02:53:57,265 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,42019,1682996021676' in zk 2023-05-02 02:53:57,266 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-02 02:53:57,266 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,39081,1682996021579' in zk 2023-05-02 02:53:57,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,266 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-02 02:53:57,266 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-02 02:53:57,266 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,38925,1682996021633' in zk 2023-05-02 02:53:57,267 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-02 02:53:57,267 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:57,267 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-02 02:53:57,268 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,268 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-02 02:53:57,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:53:57,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:53:57,268 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:57,268 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-02 02:53:57,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:53:57,270 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-02 02:53:57,270 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:57,270 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-02 02:53:57,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:53:57,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-02 02:53:57,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:57,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:57,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:53:57,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-02 02:53:57,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:57,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:57,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,39081,1682996021579': 2023-05-02 02:53:57,274 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-05-02 02:53:57,274 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-05-02 02:53:57,274 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-02 02:53:57,274 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-02 02:53:57,274 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-05-02 02:53:57,274 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-02 02:53:57,276 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,276 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,276 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:57,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:53:57,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:53:57,276 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,276 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,276 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,276 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,276 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:57,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,277 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:57,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:53:57,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,276 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,276 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:57,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,277 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:57,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:57,277 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:57,277 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:57,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:57,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,277 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:57,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:57,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-02 02:53:57,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,278 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:53:57,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-02 02:53:57,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:57,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:57,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,280 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:57,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:53:57,281 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:57,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-02 02:53:57,281 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:57,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:57,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,284 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:57,284 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:57,284 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:57,284 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,284 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:57,284 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:57,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:57,285 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:57,285 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-05-02 02:53:57,285 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-02 02:53:57,285 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:57,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:57,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:57,284 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-02 02:53:57,285 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:57,285 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:57,285 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-02 02:53:57,285 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,285 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-05-02 02:53:57,284 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:57,286 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:57,284 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:57,285 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-02 02:53:57,285 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,286 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:57,286 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:57,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:57,286 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:57,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:57,286 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:57,286 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,286 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,286 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:57,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:57,386 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-02 02:53:57,387 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-02 02:53:57,406 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-02 02:53:57,407 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-05-02 02:53:57,408 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-05-02 02:53:57,408 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-05-02 02:53:57,408 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:57,408 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-05-02 02:53:57,408 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-05-02 02:53:57,408 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,408 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-02 02:53:57,410 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:57,410 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:57,410 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:57,410 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:57,410 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:57,410 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:57,410 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:57,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:57,410 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,410 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-02 02:53:57,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:57,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-02 02:53:57,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-02 02:53:57,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-02 02:53:57,411 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,411 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,411 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:53:57,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:53:57,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-02 02:53:57,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:53:57,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-02 02:53:57,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-02 02:53:57,412 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-02 02:53:57,412 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-02 02:53:57,412 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-02 02:53:57,412 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-05-02 02:53:57,412 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-05-02 02:53:57,412 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-02 02:53:57,413 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-05-02 02:53:57,412 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-02 02:53:57,413 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-02 02:53:57,413 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-02 02:53:57,413 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-02 02:53:57,413 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:57,413 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-02 02:53:57,413 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:57,413 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-02 02:53:57,413 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:57,413 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-02 02:53:57,413 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-02 02:53:57,413 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38925,1682996021633' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-02 02:53:57,413 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-02 02:53:57,413 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-02 02:53:57,413 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42019,1682996021676' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-02 02:53:57,414 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-02 02:53:57,414 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-02 02:53:57,414 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a. 2023-05-02 02:53:57,414 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a. started... 2023-05-02 02:53:57,414 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 42fc9cc7beccd63fc2605b5f0fb93a8a 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-02 02:53:57,415 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,416 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,416 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,416 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:53:57,418 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,418 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:53:57,434 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/.tmp/cf1/286c38e805394011a004447e311f27a7 2023-05-02 02:53:57,450 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/.tmp/cf2/27e7a682d2094cedbac73b1eaab3471c 2023-05-02 02:53:57,456 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/.tmp/cf1/286c38e805394011a004447e311f27a7 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/286c38e805394011a004447e311f27a7 2023-05-02 02:53:57,461 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/286c38e805394011a004447e311f27a7, entries=10, sequenceid=19, filesize=5.0 K 2023-05-02 02:53:57,462 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/.tmp/cf2/27e7a682d2094cedbac73b1eaab3471c as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/27e7a682d2094cedbac73b1eaab3471c 2023-05-02 02:53:57,467 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/27e7a682d2094cedbac73b1eaab3471c, entries=10, sequenceid=19, filesize=5.0 K 2023-05-02 02:53:57,468 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 42fc9cc7beccd63fc2605b5f0fb93a8a in 54ms, sequenceid=19, compaction requested=false 2023-05-02 02:53:57,468 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 42fc9cc7beccd63fc2605b5f0fb93a8a: 2023-05-02 02:53:57,468 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a. 2023-05-02 02:53:57,468 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-02 02:53:57,468 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-02 02:53:57,469 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,469 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-02 02:53:57,469 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-02 02:53:57,470 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,470 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:53:57,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:53:57,471 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,471 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:53:57,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:53:57,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:53:57,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-02 02:53:57,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:57,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:57,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:53:57,473 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-05-02 02:53:57,473 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@100f324f[Count = 0] remaining members to acquire global barrier 2023-05-02 02:53:57,473 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-05-02 02:53:57,473 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,474 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,474 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,474 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,474 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,474 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,474 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,474 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-02 02:53:57,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,475 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-02 02:53:57,475 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-02 02:53:57,474 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-02 02:53:57,474 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,475 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-02 02:53:57,475 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,39081,1682996021579' in zk 2023-05-02 02:53:57,475 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,42019,1682996021676' in zk 2023-05-02 02:53:57,475 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-02 02:53:57,475 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-02 02:53:57,475 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,38925,1682996021633' in zk 2023-05-02 02:53:57,477 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-02 02:53:57,477 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:57,477 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-02 02:53:57,483 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,483 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-02 02:53:57,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:53:57,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:53:57,483 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-02 02:53:57,483 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:57,483 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:57,483 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-02 02:53:57,483 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-02 02:53:57,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:53:57,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:53:57,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-02 02:53:57,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:57,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:57,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:53:57,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-02 02:53:57,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:57,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:57,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,39081,1682996021579': 2023-05-02 02:53:57,486 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-05-02 02:53:57,487 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-05-02 02:53:57,487 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-02 02:53:57,487 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-02 02:53:57,487 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-05-02 02:53:57,487 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-02 02:53:57,488 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,488 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,488 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:57,488 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,488 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,488 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,488 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,488 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:57,488 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:57,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:53:57,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:53:57,488 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,489 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:57,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,489 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:57,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:53:57,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:57,489 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:57,489 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:57,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:57,489 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:57,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:57,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,490 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-02 02:53:57,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:53:57,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-02 02:53:57,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:57,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:57,491 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:57,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,492 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:57,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:53:57,492 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-02 02:53:57,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:57,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:57,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,496 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,496 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:57,496 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-02 02:53:57,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:57,496 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:57,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:57,496 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:57,496 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:57,496 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:57,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:57,496 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:57,496 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-05-02 02:53:57,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:57,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:57,496 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:57,496 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-02 02:53:57,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:57,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:57,496 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:57,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:57,497 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:57,497 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:57,497 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-05-02 02:53:57,497 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:57,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:57,497 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-02 02:53:57,497 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-02 02:53:57,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:57,497 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,497 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,497 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:57,497 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:57,497 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,498 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,597 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-02 02:53:57,598 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-02 02:53:57,613 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-02 02:53:57,614 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-05-02 02:53:57,615 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-05-02 02:53:57,615 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-05-02 02:53:57,615 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:57,615 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-05-02 02:53:57,615 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-05-02 02:53:57,615 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,615 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-02 02:53:57,616 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:57,616 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:57,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:57,616 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:57,616 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,617 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:57,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:57,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:57,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:57,617 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:57,617 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-02 02:53:57,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-02 02:53:57,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-02 02:53:57,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-02 02:53:57,617 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,617 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,618 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:53:57,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-02 02:53:57,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:53:57,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-02 02:53:57,618 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-02 02:53:57,618 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-02 02:53:57,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:53:57,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-02 02:53:57,618 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-02 02:53:57,618 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-05-02 02:53:57,618 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-02 02:53:57,618 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-05-02 02:53:57,618 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-05-02 02:53:57,618 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-02 02:53:57,618 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-02 02:53:57,618 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-02 02:53:57,618 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:57,618 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:57,618 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-02 02:53:57,619 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-02 02:53:57,618 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-02 02:53:57,619 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42019,1682996021676' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-02 02:53:57,619 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-02 02:53:57,619 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:57,619 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-02 02:53:57,619 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-02 02:53:57,619 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-02 02:53:57,619 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38925,1682996021633' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-02 02:53:57,619 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-02 02:53:57,619 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-02 02:53:57,619 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a. 2023-05-02 02:53:57,619 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a. started... 2023-05-02 02:53:57,619 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 42fc9cc7beccd63fc2605b5f0fb93a8a 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-02 02:53:57,620 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,621 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,621 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,621 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:53:57,621 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,621 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:53:57,630 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/.tmp/cf1/f5d9f2a618634a6e94177f02612aceaf 2023-05-02 02:53:57,646 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/.tmp/cf2/2323e300af8249a9bce7f9ed4c4f6fe1 2023-05-02 02:53:57,652 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/.tmp/cf1/f5d9f2a618634a6e94177f02612aceaf as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/f5d9f2a618634a6e94177f02612aceaf 2023-05-02 02:53:57,656 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/f5d9f2a618634a6e94177f02612aceaf, entries=10, sequenceid=32, filesize=5.0 K 2023-05-02 02:53:57,657 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/.tmp/cf2/2323e300af8249a9bce7f9ed4c4f6fe1 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/2323e300af8249a9bce7f9ed4c4f6fe1 2023-05-02 02:53:57,661 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/2323e300af8249a9bce7f9ed4c4f6fe1, entries=10, sequenceid=32, filesize=5.0 K 2023-05-02 02:53:57,662 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 42fc9cc7beccd63fc2605b5f0fb93a8a in 43ms, sequenceid=32, compaction requested=true 2023-05-02 02:53:57,662 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 42fc9cc7beccd63fc2605b5f0fb93a8a: 2023-05-02 02:53:57,662 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a. 2023-05-02 02:53:57,662 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-02 02:53:57,662 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-02 02:53:57,662 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,662 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-02 02:53:57,662 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-02 02:53:57,665 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,665 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:53:57,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:53:57,665 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,665 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:53:57,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:53:57,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:53:57,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-02 02:53:57,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:57,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:57,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:53:57,668 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-05-02 02:53:57,668 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2d43e590[Count = 0] remaining members to acquire global barrier 2023-05-02 02:53:57,668 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-05-02 02:53:57,668 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,669 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,669 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,669 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,669 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,669 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,669 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-02 02:53:57,669 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,669 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,669 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-02 02:53:57,669 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-02 02:53:57,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,669 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,38925,1682996021633' in zk 2023-05-02 02:53:57,669 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-02 02:53:57,669 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-02 02:53:57,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,670 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,39081,1682996021579' in zk 2023-05-02 02:53:57,670 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-02 02:53:57,670 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-02 02:53:57,670 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,42019,1682996021676' in zk 2023-05-02 02:53:57,674 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-02 02:53:57,674 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,674 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:57,674 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-02 02:53:57,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:53:57,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:53:57,675 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-02 02:53:57,675 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:57,675 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-02 02:53:57,677 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-02 02:53:57,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:53:57,677 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:57,677 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-02 02:53:57,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:53:57,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-02 02:53:57,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:57,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:57,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:53:57,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-02 02:53:57,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:57,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:57,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,39081,1682996021579': 2023-05-02 02:53:57,680 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-05-02 02:53:57,680 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-05-02 02:53:57,680 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-02 02:53:57,680 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-02 02:53:57,680 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-05-02 02:53:57,680 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-02 02:53:57,682 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,682 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,682 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:53:57,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:53:57,682 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:57,682 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,682 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,682 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,682 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:53:57,682 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:57,682 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:57,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,682 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:57,682 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:57,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:57,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-02 02:53:57,683 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:57,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:57,683 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:57,683 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:57,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:57,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:53:57,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,683 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-02 02:53:57,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:57,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:57,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:53:57,685 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:57,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-02 02:53:57,686 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:57,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:57,686 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:57,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,691 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,691 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:57,691 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-02 02:53:57,691 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:57,691 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:57,691 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:57,691 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:57,691 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-05-02 02:53:57,692 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-02 02:53:57,691 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:57,691 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:57,691 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:57,691 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:57,691 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:57,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:57,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:57,692 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-02 02:53:57,691 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:57,692 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,691 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:57,692 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:57,692 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:57,692 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-05-02 02:53:57,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:57,692 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:57,692 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:57,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:57,692 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:57,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:57,692 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-02 02:53:57,693 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:57,693 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,693 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,793 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-02 02:53:57,794 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-02 02:53:57,810 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-02 02:53:57,812 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-05-02 02:53:57,812 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-05-02 02:53:57,812 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-05-02 02:53:57,812 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:57,812 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-05-02 02:53:57,812 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-05-02 02:53:57,813 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,813 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-02 02:53:57,815 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:57,815 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,815 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:57,815 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:57,815 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:57,815 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:57,815 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:57,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:57,815 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,815 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-02 02:53:57,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:57,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:57,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-02 02:53:57,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-02 02:53:57,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-02 02:53:57,816 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,816 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,816 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:53:57,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-02 02:53:57,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:53:57,816 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-02 02:53:57,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:53:57,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-02 02:53:57,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-02 02:53:57,817 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-02 02:53:57,817 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-02 02:53:57,817 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-05-02 02:53:57,817 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-02 02:53:57,817 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-02 02:53:57,817 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:57,817 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-05-02 02:53:57,817 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-02 02:53:57,817 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-05-02 02:53:57,817 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-02 02:53:57,817 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-02 02:53:57,817 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-02 02:53:57,817 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-02 02:53:57,817 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:57,817 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-02 02:53:57,817 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-02 02:53:57,817 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-02 02:53:57,817 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42019,1682996021676' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-02 02:53:57,817 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:57,817 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38925,1682996021633' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-02 02:53:57,817 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-02 02:53:57,818 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-02 02:53:57,818 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-02 02:53:57,818 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a. 2023-05-02 02:53:57,818 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a. started... 2023-05-02 02:53:57,818 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 42fc9cc7beccd63fc2605b5f0fb93a8a 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-02 02:53:57,819 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,819 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,819 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,819 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:53:57,820 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,820 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:53:57,832 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/.tmp/cf1/b2cc6c2d76904b22a26933b36d566d84 2023-05-02 02:53:57,845 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/.tmp/cf2/656e2d9013c348d0a621f1369b58a6b1 2023-05-02 02:53:57,850 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/.tmp/cf1/b2cc6c2d76904b22a26933b36d566d84 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/b2cc6c2d76904b22a26933b36d566d84 2023-05-02 02:53:57,855 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/b2cc6c2d76904b22a26933b36d566d84, entries=10, sequenceid=45, filesize=5.0 K 2023-05-02 02:53:57,856 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/.tmp/cf2/656e2d9013c348d0a621f1369b58a6b1 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/656e2d9013c348d0a621f1369b58a6b1 2023-05-02 02:53:57,861 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/656e2d9013c348d0a621f1369b58a6b1, entries=10, sequenceid=45, filesize=5.0 K 2023-05-02 02:53:57,862 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 42fc9cc7beccd63fc2605b5f0fb93a8a in 44ms, sequenceid=45, compaction requested=true 2023-05-02 02:53:57,862 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 42fc9cc7beccd63fc2605b5f0fb93a8a: 2023-05-02 02:53:57,862 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a. 2023-05-02 02:53:57,862 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-02 02:53:57,862 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-02 02:53:57,862 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,862 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-02 02:53:57,862 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-02 02:53:57,864 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,864 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:53:57,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:53:57,864 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:53:57,864 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:53:57,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:53:57,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-02 02:53:57,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:57,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:57,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:53:57,866 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-05-02 02:53:57,866 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@701d1573[Count = 0] remaining members to acquire global barrier 2023-05-02 02:53:57,866 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-05-02 02:53:57,866 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,868 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,868 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,868 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,868 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,868 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,868 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-02 02:53:57,868 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,868 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-02 02:53:57,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,868 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-02 02:53:57,869 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-02 02:53:57,869 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,39081,1682996021579' in zk 2023-05-02 02:53:57,868 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,869 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-02 02:53:57,869 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,38925,1682996021633' in zk 2023-05-02 02:53:57,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,869 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-02 02:53:57,869 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-02 02:53:57,869 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,42019,1682996021676' in zk 2023-05-02 02:53:57,870 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,870 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-02 02:53:57,870 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:57,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:53:57,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:53:57,870 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-02 02:53:57,871 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-02 02:53:57,871 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:57,871 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-02 02:53:57,871 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-02 02:53:57,871 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:57,871 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-02 02:53:57,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:53:57,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:53:57,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-02 02:53:57,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:57,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:57,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:53:57,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-02 02:53:57,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:57,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:57,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,39081,1682996021579': 2023-05-02 02:53:57,874 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-05-02 02:53:57,874 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-05-02 02:53:57,874 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-02 02:53:57,874 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-02 02:53:57,874 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-05-02 02:53:57,874 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-02 02:53:57,876 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,876 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:57,876 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,876 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,876 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,876 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,876 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:53:57,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:53:57,876 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:57,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,876 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,876 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:57,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,877 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:57,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:57,877 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:57,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:53:57,877 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:57,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:57,877 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:57,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:57,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,877 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:57,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-02 02:53:57,878 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:53:57,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-02 02:53:57,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:57,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:57,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:53:57,880 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:57,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-02 02:53:57,880 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:57,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:57,880 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:57,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,884 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,884 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:57,884 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-02 02:53:57,884 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:57,884 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:57,884 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:57,884 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:57,884 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:57,884 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:57,884 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:57,884 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:57,884 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-02 02:53:57,884 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:57,884 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-05-02 02:53:57,885 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-02 02:53:57,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:57,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:57,884 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:57,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:57,885 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:57,885 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-05-02 02:53:57,884 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:57,885 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-02 02:53:57,884 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:57,885 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,885 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:57,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:57,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:57,885 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:57,885 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:57,886 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:57,886 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:57,885 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:57,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:57,985 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-02 02:53:57,986 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-02 02:53:58,002 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-02 02:53:58,004 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-05-02 02:53:58,004 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-05-02 02:53:58,004 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-05-02 02:53:58,004 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:58,004 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-05-02 02:53:58,004 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-05-02 02:53:58,005 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:58,005 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-02 02:53:58,006 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:58,006 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:58,006 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:58,006 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:58,006 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:58,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:58,006 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:58,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:58,006 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:58,006 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-02 02:53:58,006 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:58,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:58,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-02 02:53:58,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-02 02:53:58,007 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:58,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-02 02:53:58,007 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:58,007 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:58,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:53:58,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:53:58,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-02 02:53:58,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-02 02:53:58,008 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-02 02:53:58,008 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-02 02:53:58,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:53:58,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-02 02:53:58,008 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-02 02:53:58,008 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-05-02 02:53:58,008 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-02 02:53:58,008 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-05-02 02:53:58,008 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-02 02:53:58,008 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-05-02 02:53:58,008 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-02 02:53:58,008 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-02 02:53:58,008 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:58,008 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-02 02:53:58,008 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-02 02:53:58,009 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:58,009 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-02 02:53:58,009 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-02 02:53:58,009 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:58,009 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-02 02:53:58,009 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42019,1682996021676' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-02 02:53:58,009 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-02 02:53:58,009 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-02 02:53:58,009 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-02 02:53:58,009 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a. 2023-05-02 02:53:58,009 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-02 02:53:58,009 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a. started... 2023-05-02 02:53:58,009 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38925,1682996021633' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-02 02:53:58,009 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 42fc9cc7beccd63fc2605b5f0fb93a8a 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-02 02:53:58,010 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:58,011 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:58,011 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:58,011 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:53:58,011 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:58,011 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:53:58,020 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/.tmp/cf1/b7760caf241143f898d4aba7b39f755e 2023-05-02 02:53:58,037 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/.tmp/cf2/f6c6cd94aaf94c469cc58db060f94145 2023-05-02 02:53:58,042 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/.tmp/cf1/b7760caf241143f898d4aba7b39f755e as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/b7760caf241143f898d4aba7b39f755e 2023-05-02 02:53:58,046 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/b7760caf241143f898d4aba7b39f755e, entries=10, sequenceid=58, filesize=5.0 K 2023-05-02 02:53:58,047 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/.tmp/cf2/f6c6cd94aaf94c469cc58db060f94145 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/f6c6cd94aaf94c469cc58db060f94145 2023-05-02 02:53:58,051 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/f6c6cd94aaf94c469cc58db060f94145, entries=10, sequenceid=58, filesize=5.0 K 2023-05-02 02:53:58,052 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 42fc9cc7beccd63fc2605b5f0fb93a8a in 43ms, sequenceid=58, compaction requested=true 2023-05-02 02:53:58,052 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 42fc9cc7beccd63fc2605b5f0fb93a8a: 2023-05-02 02:53:58,052 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a. 2023-05-02 02:53:58,052 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-02 02:53:58,052 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-02 02:53:58,052 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:58,052 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-02 02:53:58,052 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-02 02:53:58,055 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:58,055 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:58,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:58,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:53:58,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:53:58,056 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:58,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:53:58,056 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:53:58,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:53:58,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-02 02:53:58,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:58,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:58,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:58,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:53:58,057 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-05-02 02:53:58,057 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2befebdd[Count = 0] remaining members to acquire global barrier 2023-05-02 02:53:58,057 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-05-02 02:53:58,058 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:58,059 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:58,059 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:58,059 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:58,059 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:58,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:58,059 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:58,059 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-02 02:53:58,059 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:58,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:58,059 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:58,059 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-02 02:53:58,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:58,059 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-02 02:53:58,059 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-02 02:53:58,059 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-02 02:53:58,059 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,42019,1682996021676' in zk 2023-05-02 02:53:58,059 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-02 02:53:58,059 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-02 02:53:58,059 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,39081,1682996021579' in zk 2023-05-02 02:53:58,059 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,38925,1682996021633' in zk 2023-05-02 02:53:58,061 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-02 02:53:58,061 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:58,061 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-02 02:53:58,065 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-02 02:53:58,065 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-02 02:53:58,065 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:58,065 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:58,065 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-02 02:53:58,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:58,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:53:58,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:53:58,065 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:58,066 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-02 02:53:58,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:53:58,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:53:58,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-02 02:53:58,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:58,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:58,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:58,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:53:58,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-02 02:53:58,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:58,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:58,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:58,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,39081,1682996021579': 2023-05-02 02:53:58,069 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-05-02 02:53:58,069 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-05-02 02:53:58,069 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-02 02:53:58,069 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-02 02:53:58,069 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-05-02 02:53:58,069 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-02 02:53:58,070 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:58,070 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:58,070 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:58,070 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:58,070 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:58,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:58,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:53:58,070 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:58,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:53:58,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:58,070 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:58,070 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:58,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:58,071 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:58,071 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:58,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:58,071 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:58,071 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:58,071 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:58,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:58,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:53:58,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:58,072 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:58,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:58,072 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:58,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:58,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:58,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:58,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-02 02:53:58,077 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:58,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:53:58,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-02 02:53:58,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:58,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:58,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:58,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:53:58,079 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:58,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-02 02:53:58,079 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:58,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:58,079 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:58,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:58,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:58,082 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:58,082 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:58,082 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:58,082 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:58,082 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:58,082 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:58,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:58,082 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:58,082 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:58,082 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-02 02:53:58,083 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:58,083 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:58,083 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:58,083 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:58,082 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-05-02 02:53:58,083 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-02 02:53:58,082 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:58,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:58,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:58,083 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:58,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:58,083 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-05-02 02:53:58,083 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-02 02:53:58,083 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-02 02:53:58,083 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:58,083 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:58,083 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:58,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:58,083 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:58,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:58,084 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:58,084 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:58,084 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:58,084 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:58,183 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-02 02:53:58,184 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-02 02:53:58,199 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-02 02:53:58,201 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-05-02 02:53:58,201 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-05-02 02:53:58,201 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-05-02 02:53:58,201 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:58,201 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-05-02 02:53:58,201 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-05-02 02:53:58,202 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:58,202 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-02 02:53:58,203 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:58,203 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:58,203 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:58,203 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:58,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:58,203 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:58,203 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:58,203 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:58,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:58,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:58,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-02 02:53:58,204 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:58,204 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-02 02:53:58,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-02 02:53:58,204 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:58,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-02 02:53:58,204 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:58,204 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:58,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:53:58,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-02 02:53:58,204 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-02 02:53:58,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:53:58,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-02 02:53:58,205 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-02 02:53:58,205 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-05-02 02:53:58,205 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-02 02:53:58,205 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-05-02 02:53:58,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:53:58,205 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-02 02:53:58,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-02 02:53:58,205 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-02 02:53:58,205 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:58,205 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-02 02:53:58,205 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-02 02:53:58,205 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-02 02:53:58,205 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-02 02:53:58,205 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:58,205 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42019,1682996021676' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-02 02:53:58,205 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-02 02:53:58,205 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-02 02:53:58,205 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-05-02 02:53:58,205 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38925,1682996021633' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-02 02:53:58,206 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-02 02:53:58,206 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-02 02:53:58,206 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:53:58,206 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-02 02:53:58,206 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-02 02:53:58,206 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-02 02:53:58,206 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a. 2023-05-02 02:53:58,206 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a. started... 2023-05-02 02:53:58,206 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 42fc9cc7beccd63fc2605b5f0fb93a8a 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-02 02:53:58,208 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:58,209 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:58,209 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:58,209 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:53:58,209 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:58,209 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:53:58,216 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/.tmp/cf1/bd22d3c0cbb241fabad930e22fe8b28d 2023-05-02 02:53:58,232 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/.tmp/cf2/477cc447542345f2a675a41d3cb656df 2023-05-02 02:53:58,237 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/.tmp/cf1/bd22d3c0cbb241fabad930e22fe8b28d as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/bd22d3c0cbb241fabad930e22fe8b28d 2023-05-02 02:53:58,242 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/bd22d3c0cbb241fabad930e22fe8b28d, entries=10, sequenceid=71, filesize=5.0 K 2023-05-02 02:53:58,242 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/.tmp/cf2/477cc447542345f2a675a41d3cb656df as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/477cc447542345f2a675a41d3cb656df 2023-05-02 02:53:58,247 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/477cc447542345f2a675a41d3cb656df, entries=10, sequenceid=71, filesize=5.0 K 2023-05-02 02:53:58,248 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 42fc9cc7beccd63fc2605b5f0fb93a8a in 42ms, sequenceid=71, compaction requested=true 2023-05-02 02:53:58,248 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 42fc9cc7beccd63fc2605b5f0fb93a8a: 2023-05-02 02:53:58,248 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a. 2023-05-02 02:53:58,248 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-02 02:53:58,248 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-02 02:53:58,248 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:58,248 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-02 02:53:58,248 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-02 02:53:58,250 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:58,250 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:58,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:58,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:53:58,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:53:58,250 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:58,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:53:58,250 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:53:58,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:53:58,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-02 02:53:58,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:58,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:58,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:58,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:53:58,252 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-05-02 02:53:58,252 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@421b39a5[Count = 0] remaining members to acquire global barrier 2023-05-02 02:53:58,252 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-05-02 02:53:58,253 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:58,253 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:58,253 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:58,253 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:58,254 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:58,253 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:58,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:58,254 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:58,254 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-02 02:53:58,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:58,254 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:58,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:58,254 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-02 02:53:58,254 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-02 02:53:58,254 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-02 02:53:58,254 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-02 02:53:58,254 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-02 02:53:58,254 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,39081,1682996021579' in zk 2023-05-02 02:53:58,254 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-02 02:53:58,254 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,42019,1682996021676' in zk 2023-05-02 02:53:58,254 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,38925,1682996021633' in zk 2023-05-02 02:53:58,257 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:58,257 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-02 02:53:58,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:58,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:53:58,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:53:58,257 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:58,257 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-02 02:53:58,257 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-02 02:53:58,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:53:58,257 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:58,257 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-02 02:53:58,257 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:58,257 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-02 02:53:58,257 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-02 02:53:58,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:53:58,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-02 02:53:58,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:58,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:58,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:58,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:53:58,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-02 02:53:58,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:58,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:58,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:58,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,39081,1682996021579': 2023-05-02 02:53:58,260 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-05-02 02:53:58,260 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-05-02 02:53:58,260 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-02 02:53:58,260 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-02 02:53:58,260 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-05-02 02:53:58,261 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-02 02:53:58,262 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:58,262 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:58,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:58,262 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:58,262 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:58,262 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:58,262 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:58,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:58,262 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:58,262 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:58,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:53:58,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:53:58,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:58,262 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:58,262 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:58,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:58,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:58,263 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:58,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:58,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:58,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:58,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:58,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:58,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:53:58,264 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:58,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:58,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:58,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-02 02:53:58,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:58,264 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:58,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:53:58,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-02 02:53:58,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:58,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:58,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:58,266 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:58,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:53:58,266 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:58,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-02 02:53:58,266 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:58,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:58,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:58,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:58,273 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:58,273 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:58,273 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:58,273 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:53:58,273 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:58,273 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-02 02:53:58,273 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:53:58,273 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:58,273 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:58,273 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:58,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:58,273 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:53:58,273 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:53:58,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:58,273 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:58,273 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-05-02 02:53:58,273 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-02 02:53:58,273 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:58,273 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:58,274 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:58,274 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-02 02:53:58,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:53:58,274 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:58,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:58,274 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:58,274 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-05-02 02:53:58,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:58,274 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-02 02:53:58,274 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:53:58,274 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:53:58,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:53:58,274 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:53:58,275 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-02 02:53:58,275 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-02 02:53:58,374 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-02 02:53:58,376 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-02 02:53:58,636 WARN [Listener at localhost/37119] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-05-02 02:53:58,638 DEBUG [Listener at localhost/37119] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:58,639 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(1029): Stored pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:53:58,641 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:53:58,641 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:53:58,641 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:53:58,642 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-02 02:53:58,642 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:53:58,642 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-02 02:53:58,642 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-02 02:53:58,890 INFO [Listener at localhost/37119] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:53:58,890 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-02 02:53:58,890 INFO [Listener at localhost/37119] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-02 02:53:58,890 INFO [Listener at localhost/37119] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-02 02:53:58,890 ERROR [Listener at localhost/37119] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-7,5,PEWorkerGroup] Thread[HFileArchiver-8,5,PEWorkerGroup] 2023-05-02 02:53:58,890 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-02 02:53:58,890 INFO [Listener at localhost/37119] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-02 02:53:58,890 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-02 02:53:58,890 INFO [Listener at localhost/37119] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-02 02:53:58,891 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-02 02:53:58,892 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-05-02 02:53:58,893 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-02 02:53:58,893 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-02 02:53:58,893 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-02 02:53:58,894 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-02 02:53:58,894 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:53:58,894 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=084b1607f7ed8615120a04e4729fbafa, daughterA=ea7683cd3436a62eb16418a3c875d563, daughterB=408e891cab9b919f4038b233ba34779d 2023-05-02 02:53:58,895 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:53:58,895 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:53:58,895 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-02 02:53:58,895 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:58,895 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-02 02:53:58,896 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-02 02:53:58,896 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-02 02:53:58,896 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:53:58,896 DEBUG [Listener at localhost/37119] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 held the lock before restarting, call acquireLock to restore it. 2023-05-02 02:53:58,896 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:53:58,896 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:53:58,897 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:53:58,897 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-05-02 02:53:58,897 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-02 02:53:58,898 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,38925,1682996021633, table=hbase:meta, region=1588230740 2023-05-02 02:53:58,898 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682996068898 2023-05-02 02:53:58,900 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(1517): Joining cluster... 2023-05-02 02:53:58,900 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:53:58,902 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=5c823d6f15ad7b8fe82f9ae4beee6513, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42019,1682996021676, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676, openSeqNum=2 2023-05-02 02:53:58,903 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=42fc9cc7beccd63fc2605b5f0fb93a8a, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39081,1682996021579, regionLocation=jenkins-hbase4.apache.org,39081,1682996021579, openSeqNum=2 2023-05-02 02:53:58,903 INFO [Listener at localhost/37119] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-02 02:53:58,903 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682996098903 2023-05-02 02:53:58,903 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682996158903 2023-05-02 02:53:58,903 INFO [Listener at localhost/37119] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-02 02:53:58,903 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-02 02:53:58,905 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-02 02:53:58,905 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:53:58,905 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-02 02:53:58,905 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-02 02:53:59,153 INFO [Listener at localhost/37119] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:53:59,153 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-02 02:53:59,154 INFO [Listener at localhost/37119] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-02 02:53:59,154 INFO [Listener at localhost/37119] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-02 02:53:59,155 ERROR [Listener at localhost/37119] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-02 02:53:59,155 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-02 02:53:59,155 INFO [Listener at localhost/37119] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-02 02:53:59,155 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-02 02:53:59,155 INFO [Listener at localhost/37119] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-02 02:53:59,156 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-02 02:53:59,157 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-05-02 02:53:59,157 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-02 02:53:59,158 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-02 02:53:59,158 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-02 02:53:59,158 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-02 02:53:59,158 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:53:59,159 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=084b1607f7ed8615120a04e4729fbafa, daughterA=ea7683cd3436a62eb16418a3c875d563, daughterB=408e891cab9b919f4038b233ba34779d 2023-05-02 02:53:59,159 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:53:59,159 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:53:59,159 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-02 02:53:59,159 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:59,160 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-02 02:53:59,160 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-02 02:53:59,160 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-02 02:53:59,160 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:53:59,160 DEBUG [Listener at localhost/37119] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 held the lock before restarting, call acquireLock to restore it. 2023-05-02 02:53:59,160 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:53:59,161 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:53:59,161 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:53:59,161 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-05-02 02:53:59,161 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-02 02:53:59,162 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,38925,1682996021633, table=hbase:meta, region=1588230740 2023-05-02 02:53:59,162 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682996069162 2023-05-02 02:53:59,164 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:53:59,164 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(1517): Joining cluster... 2023-05-02 02:53:59,166 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=5c823d6f15ad7b8fe82f9ae4beee6513, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42019,1682996021676, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676, openSeqNum=2 2023-05-02 02:53:59,166 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=42fc9cc7beccd63fc2605b5f0fb93a8a, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39081,1682996021579, regionLocation=jenkins-hbase4.apache.org,39081,1682996021579, openSeqNum=2 2023-05-02 02:53:59,166 INFO [Listener at localhost/37119] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-02 02:53:59,166 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682996099166 2023-05-02 02:53:59,166 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682996159166 2023-05-02 02:53:59,167 INFO [Listener at localhost/37119] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-05-02 02:53:59,167 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-02 02:53:59,169 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, UNASSIGN}] 2023-05-02 02:53:59,169 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-02 02:53:59,169 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:53:59,169 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-02 02:53:59,169 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-02 02:53:59,417 INFO [Listener at localhost/37119] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:53:59,417 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-02 02:53:59,417 INFO [Listener at localhost/37119] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-02 02:53:59,417 INFO [Listener at localhost/37119] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-02 02:53:59,417 ERROR [Listener at localhost/37119] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-02 02:53:59,417 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-02 02:53:59,417 INFO [Listener at localhost/37119] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-02 02:53:59,417 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-02 02:53:59,418 INFO [Listener at localhost/37119] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-02 02:53:59,418 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-02 02:53:59,420 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-05-02 02:53:59,420 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-02 02:53:59,421 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-02 02:53:59,421 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-02 02:53:59,421 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-02 02:53:59,422 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:53:59,422 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=084b1607f7ed8615120a04e4729fbafa, daughterA=ea7683cd3436a62eb16418a3c875d563, daughterB=408e891cab9b919f4038b233ba34779d 2023-05-02 02:53:59,422 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:53:59,423 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:53:59,423 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-02 02:53:59,424 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:59,424 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-02 02:53:59,425 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-02 02:53:59,425 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-02 02:53:59,426 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:53:59,426 DEBUG [Listener at localhost/37119] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 held the lock before restarting, call acquireLock to restore it. 2023-05-02 02:53:59,426 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:53:59,426 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:53:59,426 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:53:59,426 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-05-02 02:53:59,427 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-02 02:53:59,427 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,38925,1682996021633, table=hbase:meta, region=1588230740 2023-05-02 02:53:59,428 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682996069428 2023-05-02 02:53:59,428 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(1517): Joining cluster... 2023-05-02 02:53:59,429 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:53:59,431 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=5c823d6f15ad7b8fe82f9ae4beee6513, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42019,1682996021676, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676, openSeqNum=2 2023-05-02 02:53:59,432 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=42fc9cc7beccd63fc2605b5f0fb93a8a, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39081,1682996021579, regionLocation=jenkins-hbase4.apache.org,39081,1682996021579, openSeqNum=2 2023-05-02 02:53:59,432 INFO [Listener at localhost/37119] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-02 02:53:59,432 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682996099432 2023-05-02 02:53:59,432 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682996159432 2023-05-02 02:53:59,432 INFO [Listener at localhost/37119] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-05-02 02:53:59,432 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, UNASSIGN}] 2023-05-02 02:53:59,432 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-02 02:53:59,434 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, UNASSIGN 2023-05-02 02:53:59,435 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=42fc9cc7beccd63fc2605b5f0fb93a8a, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:59,435 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1682996039435"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1682996039435"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682996039435"}]},"ts":"1682996039435"} 2023-05-02 02:53:59,437 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 42fc9cc7beccd63fc2605b5f0fb93a8a, server=jenkins-hbase4.apache.org,39081,1682996021579}] 2023-05-02 02:53:59,437 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-02 02:53:59,437 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, UNASSIGN 2023-05-02 02:53:59,437 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-02 02:53:59,437 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-02 02:53:59,682 INFO [Listener at localhost/37119] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:53:59,682 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-02 02:53:59,682 INFO [Listener at localhost/37119] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-02 02:53:59,682 INFO [Listener at localhost/37119] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-02 02:53:59,684 ERROR [Listener at localhost/37119] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-02 02:53:59,684 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-02 02:53:59,684 INFO [Listener at localhost/37119] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-02 02:53:59,684 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-02 02:53:59,684 INFO [Listener at localhost/37119] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-02 02:53:59,685 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-02 02:53:59,686 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-05-02 02:53:59,686 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-05-02 02:53:59,686 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-02 02:53:59,686 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-02 02:53:59,687 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-02 02:53:59,687 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-02 02:53:59,687 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:53:59,687 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=084b1607f7ed8615120a04e4729fbafa, daughterA=ea7683cd3436a62eb16418a3c875d563, daughterB=408e891cab9b919f4038b233ba34779d 2023-05-02 02:53:59,688 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:53:59,688 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:53:59,688 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-02 02:53:59,688 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:59,688 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-02 02:53:59,689 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-02 02:53:59,689 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-02 02:53:59,689 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:53:59,690 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, UNASSIGN 2023-05-02 02:53:59,690 DEBUG [Listener at localhost/37119] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 held the lock before restarting, call acquireLock to restore it. 2023-05-02 02:53:59,690 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:53:59,690 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:53:59,690 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:53:59,690 DEBUG [Listener at localhost/37119] procedure2.Procedure(966): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-05-02 02:53:59,690 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, UNASSIGN 2023-05-02 02:53:59,690 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-05-02 02:53:59,691 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-02 02:53:59,692 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,38925,1682996021633, table=hbase:meta, region=1588230740 2023-05-02 02:53:59,692 INFO [Listener at localhost/37119] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a to restore RIT 2023-05-02 02:53:59,696 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682996069696 2023-05-02 02:53:59,697 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, UNASSIGN 2023-05-02 02:53:59,697 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(1517): Joining cluster... 2023-05-02 02:53:59,699 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=5c823d6f15ad7b8fe82f9ae4beee6513, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42019,1682996021676, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676, openSeqNum=2 2023-05-02 02:53:59,700 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=42fc9cc7beccd63fc2605b5f0fb93a8a, regionState=CLOSING, lastHost=jenkins-hbase4.apache.org,39081,1682996021579, regionLocation=jenkins-hbase4.apache.org,39081,1682996021579, openSeqNum=2 2023-05-02 02:53:59,700 INFO [Listener at localhost/37119] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-02 02:53:59,700 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682996099700 2023-05-02 02:53:59,700 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682996159700 2023-05-02 02:53:59,700 INFO [Listener at localhost/37119] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-02 02:53:59,700 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=42fc9cc7beccd63fc2605b5f0fb93a8a, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:59,701 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1682996039700"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1682996039700"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682996039700"}]},"ts":"1682996039700"} 2023-05-02 02:53:59,702 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 42fc9cc7beccd63fc2605b5f0fb93a8a, server=jenkins-hbase4.apache.org,39081,1682996021579}] 2023-05-02 02:53:59,702 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-02 02:53:59,856 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:53:59,856 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 42fc9cc7beccd63fc2605b5f0fb93a8a, disabling compactions & flushes 2023-05-02 02:53:59,856 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a. 2023-05-02 02:53:59,856 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a. 2023-05-02 02:53:59,856 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a. after waiting 0 ms 2023-05-02 02:53:59,856 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a. 2023-05-02 02:53:59,856 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 42fc9cc7beccd63fc2605b5f0fb93a8a 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-05-02 02:53:59,866 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/.tmp/cf1/e715c20fa2c6439d88c153d1aef48552 2023-05-02 02:53:59,883 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/.tmp/cf2/51d64a94e5f64ff684e29f64ff7eaec6 2023-05-02 02:53:59,888 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/.tmp/cf1/e715c20fa2c6439d88c153d1aef48552 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/e715c20fa2c6439d88c153d1aef48552 2023-05-02 02:53:59,893 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/e715c20fa2c6439d88c153d1aef48552, entries=8, sequenceid=82, filesize=5.0 K 2023-05-02 02:53:59,894 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/.tmp/cf2/51d64a94e5f64ff684e29f64ff7eaec6 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/51d64a94e5f64ff684e29f64ff7eaec6 2023-05-02 02:53:59,899 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/51d64a94e5f64ff684e29f64ff7eaec6, entries=8, sequenceid=82, filesize=5.0 K 2023-05-02 02:53:59,899 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 42fc9cc7beccd63fc2605b5f0fb93a8a in 43ms, sequenceid=82, compaction requested=true 2023-05-02 02:53:59,905 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-05-02 02:53:59,905 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-02 02:53:59,905 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a. 2023-05-02 02:53:59,905 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 42fc9cc7beccd63fc2605b5f0fb93a8a: 2023-05-02 02:53:59,908 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:53:59,908 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=42fc9cc7beccd63fc2605b5f0fb93a8a, regionState=CLOSED 2023-05-02 02:53:59,909 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1682996039908"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682996039908"}]},"ts":"1682996039908"} 2023-05-02 02:53:59,910 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-02 02:53:59,910 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure 42fc9cc7beccd63fc2605b5f0fb93a8a, server=jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:59,910 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-02 02:53:59,910 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure 42fc9cc7beccd63fc2605b5f0fb93a8a, server=jenkins-hbase4.apache.org,39081,1682996021579 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-02 02:53:59,950 INFO [Listener at localhost/37119] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:53:59,951 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-02 02:53:59,951 INFO [Listener at localhost/37119] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-02 02:53:59,951 INFO [Listener at localhost/37119] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-02 02:53:59,951 ERROR [Listener at localhost/37119] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-02 02:53:59,951 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-02 02:53:59,951 INFO [Listener at localhost/37119] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-02 02:53:59,951 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-02 02:53:59,951 INFO [Listener at localhost/37119] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-02 02:53:59,952 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-02 02:53:59,953 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-05-02 02:53:59,953 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-05-02 02:53:59,953 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-05-02 02:53:59,953 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-02 02:53:59,954 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-02 02:53:59,954 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-02 02:53:59,954 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-02 02:53:59,954 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:53:59,954 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=084b1607f7ed8615120a04e4729fbafa, daughterA=ea7683cd3436a62eb16418a3c875d563, daughterB=408e891cab9b919f4038b233ba34779d 2023-05-02 02:53:59,955 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:53:59,955 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:53:59,955 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-02 02:53:59,955 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:53:59,955 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-02 02:53:59,955 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-02 02:53:59,956 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-02 02:53:59,956 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:53:59,956 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, UNASSIGN 2023-05-02 02:53:59,956 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 42fc9cc7beccd63fc2605b5f0fb93a8a, server=jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:59,957 DEBUG [Listener at localhost/37119] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 held the lock before restarting, call acquireLock to restore it. 2023-05-02 02:53:59,957 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:53:59,957 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:53:59,957 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:53:59,957 DEBUG [Listener at localhost/37119] procedure2.Procedure(966): pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-05-02 02:53:59,957 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, UNASSIGN 2023-05-02 02:53:59,957 DEBUG [Listener at localhost/37119] procedure2.Procedure(966): pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 42fc9cc7beccd63fc2605b5f0fb93a8a, server=jenkins-hbase4.apache.org,39081,1682996021579 held the lock before restarting, call acquireLock to restore it. 2023-05-02 02:53:59,957 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-05-02 02:53:59,958 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-02 02:53:59,958 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,38925,1682996021633, table=hbase:meta, region=1588230740 2023-05-02 02:53:59,958 INFO [Listener at localhost/37119] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a to restore RIT 2023-05-02 02:53:59,960 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682996069960 2023-05-02 02:53:59,960 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(1517): Joining cluster... 2023-05-02 02:53:59,961 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 42fc9cc7beccd63fc2605b5f0fb93a8a, server=jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:59,963 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=5c823d6f15ad7b8fe82f9ae4beee6513, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42019,1682996021676, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676, openSeqNum=2 2023-05-02 02:53:59,963 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=42fc9cc7beccd63fc2605b5f0fb93a8a, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,39081,1682996021579, regionLocation=jenkins-hbase4.apache.org,39081,1682996021579, openSeqNum=2 2023-05-02 02:53:59,963 INFO [Listener at localhost/37119] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-02 02:53:59,963 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682996099963 2023-05-02 02:53:59,963 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682996159963 2023-05-02 02:53:59,963 INFO [Listener at localhost/37119] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-02 02:53:59,964 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=42fc9cc7beccd63fc2605b5f0fb93a8a, regionState=CLOSED, regionLocation=jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:53:59,964 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1682996039963"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1682996039963"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682996039963"}]},"ts":"1682996039963"} 2023-05-02 02:53:59,965 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-02 02:53:59,968 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=46, resume processing ppid=45 2023-05-02 02:53:59,968 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 42fc9cc7beccd63fc2605b5f0fb93a8a, server=jenkins-hbase4.apache.org,39081,1682996021579 in 263 msec 2023-05-02 02:53:59,969 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-02 02:53:59,969 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, UNASSIGN 2023-05-02 02:53:59,969 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-02 02:53:59,969 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-02 02:54:00,214 INFO [Listener at localhost/37119] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:00,214 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-02 02:54:00,214 INFO [Listener at localhost/37119] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-02 02:54:00,214 INFO [Listener at localhost/37119] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-02 02:54:00,216 ERROR [Listener at localhost/37119] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-02 02:54:00,217 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-02 02:54:00,217 INFO [Listener at localhost/37119] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-02 02:54:00,217 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-02 02:54:00,217 INFO [Listener at localhost/37119] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-02 02:54:00,217 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-02 02:54:00,219 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-05-02 02:54:00,219 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-05-02 02:54:00,219 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-02 02:54:00,219 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-02 02:54:00,220 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-02 02:54:00,220 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-02 02:54:00,220 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-02 02:54:00,221 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:54:00,221 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=084b1607f7ed8615120a04e4729fbafa, daughterA=ea7683cd3436a62eb16418a3c875d563, daughterB=408e891cab9b919f4038b233ba34779d 2023-05-02 02:54:00,221 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:54:00,222 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:54:00,222 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-02 02:54:00,222 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:54:00,222 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-02 02:54:00,222 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-02 02:54:00,223 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-02 02:54:00,224 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:00,224 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, UNASSIGN 2023-05-02 02:54:00,224 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 42fc9cc7beccd63fc2605b5f0fb93a8a, server=jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:00,224 DEBUG [Listener at localhost/37119] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 held the lock before restarting, call acquireLock to restore it. 2023-05-02 02:54:00,224 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:00,224 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:00,224 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:00,224 DEBUG [Listener at localhost/37119] procedure2.Procedure(946): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-02 02:54:00,224 DEBUG [Listener at localhost/37119] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 42fc9cc7beccd63fc2605b5f0fb93a8a, server=jenkins-hbase4.apache.org,39081,1682996021579 didn't hold the lock before restarting, skip acquiring lock. 2023-05-02 02:54:00,225 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-05-02 02:54:00,225 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-02 02:54:00,226 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,38925,1682996021633, table=hbase:meta, region=1588230740 2023-05-02 02:54:00,226 INFO [Listener at localhost/37119] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a to restore RIT 2023-05-02 02:54:00,226 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682996070226 2023-05-02 02:54:00,227 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(1517): Joining cluster... 2023-05-02 02:54:00,227 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, UNASSIGN 2023-05-02 02:54:00,229 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=5c823d6f15ad7b8fe82f9ae4beee6513, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42019,1682996021676, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676, openSeqNum=2 2023-05-02 02:54:00,230 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=42fc9cc7beccd63fc2605b5f0fb93a8a, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,39081,1682996021579, regionLocation=jenkins-hbase4.apache.org,39081,1682996021579, openSeqNum=2 2023-05-02 02:54:00,230 INFO [Listener at localhost/37119] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-02 02:54:00,230 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682996100230 2023-05-02 02:54:00,230 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682996160230 2023-05-02 02:54:00,230 INFO [Listener at localhost/37119] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-05-02 02:54:00,230 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, UNASSIGN 2023-05-02 02:54:00,232 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-02 02:54:00,235 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=45, resume processing ppid=44 2023-05-02 02:54:00,235 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, UNASSIGN in 800 msec 2023-05-02 02:54:00,238 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-02 02:54:00,238 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:00,238 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-02 02:54:00,238 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-02 02:54:00,480 INFO [Listener at localhost/37119] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:00,481 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-02 02:54:00,481 INFO [Listener at localhost/37119] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-02 02:54:00,481 INFO [Listener at localhost/37119] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-02 02:54:00,482 ERROR [Listener at localhost/37119] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-02 02:54:00,482 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-02 02:54:00,482 INFO [Listener at localhost/37119] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-02 02:54:00,483 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-02 02:54:00,483 INFO [Listener at localhost/37119] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-02 02:54:00,483 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-02 02:54:00,484 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-05-02 02:54:00,484 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-02 02:54:00,485 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-02 02:54:00,485 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-02 02:54:00,485 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-02 02:54:00,485 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-02 02:54:00,485 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-02 02:54:00,486 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:54:00,486 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=084b1607f7ed8615120a04e4729fbafa, daughterA=ea7683cd3436a62eb16418a3c875d563, daughterB=408e891cab9b919f4038b233ba34779d 2023-05-02 02:54:00,486 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:54:00,486 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:54:00,486 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-02 02:54:00,487 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:54:00,487 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-02 02:54:00,487 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-02 02:54:00,487 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-02 02:54:00,488 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:00,488 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, UNASSIGN 2023-05-02 02:54:00,488 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 42fc9cc7beccd63fc2605b5f0fb93a8a, server=jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:00,488 DEBUG [Listener at localhost/37119] procedure2.Procedure(946): pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 didn't hold the lock before restarting, skip acquiring lock. 2023-05-02 02:54:00,488 DEBUG [Listener at localhost/37119] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-02 02:54:00,488 DEBUG [Listener at localhost/37119] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 42fc9cc7beccd63fc2605b5f0fb93a8a, server=jenkins-hbase4.apache.org,39081,1682996021579 didn't hold the lock before restarting, skip acquiring lock. 2023-05-02 02:54:00,488 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-05-02 02:54:00,496 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-02 02:54:00,497 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,38925,1682996021633, table=hbase:meta, region=1588230740 2023-05-02 02:54:00,499 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682996070499 2023-05-02 02:54:00,501 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:00,501 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(1517): Joining cluster... 2023-05-02 02:54:00,503 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=5c823d6f15ad7b8fe82f9ae4beee6513, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42019,1682996021676, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676, openSeqNum=2 2023-05-02 02:54:00,504 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=42fc9cc7beccd63fc2605b5f0fb93a8a, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,39081,1682996021579, regionLocation=jenkins-hbase4.apache.org,39081,1682996021579, openSeqNum=2 2023-05-02 02:54:00,504 INFO [Listener at localhost/37119] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-02 02:54:00,504 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682996100504 2023-05-02 02:54:00,504 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682996160504 2023-05-02 02:54:00,504 INFO [Listener at localhost/37119] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-02 02:54:00,504 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:00,504 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:00,504 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:00,509 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-02 02:54:00,517 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=42fc9cc7beccd63fc2605b5f0fb93a8a, threads=14 2023-05-02 02:54:00,518 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/2323e300af8249a9bce7f9ed4c4f6fe1 for region: 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:54:00,518 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/27e7a682d2094cedbac73b1eaab3471c for region: 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:54:00,518 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/477cc447542345f2a675a41d3cb656df for region: 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:54:00,519 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/51d64a94e5f64ff684e29f64ff7eaec6 for region: 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:54:00,519 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/65211868ead84a0895c2ebf9105167e0 for region: 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:54:00,519 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/656e2d9013c348d0a621f1369b58a6b1 for region: 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:54:00,519 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/f6c6cd94aaf94c469cc58db060f94145 for region: 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:54:00,520 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/286c38e805394011a004447e311f27a7 for region: 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:54:00,520 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/b2cc6c2d76904b22a26933b36d566d84 for region: 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:54:00,520 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/b7760caf241143f898d4aba7b39f755e for region: 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:54:00,521 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/bd22d3c0cbb241fabad930e22fe8b28d for region: 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:54:00,524 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/bde0a9d3ebfa4e1996e42b844971f197 for region: 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:54:00,524 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/e715c20fa2c6439d88c153d1aef48552 for region: 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:54:00,524 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/f5d9f2a618634a6e94177f02612aceaf for region: 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:54:00,712 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/f6c6cd94aaf94c469cc58db060f94145 for region: 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:54:00,717 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/27e7a682d2094cedbac73b1eaab3471c for region: 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:54:00,717 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/65211868ead84a0895c2ebf9105167e0 for region: 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:54:00,729 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/b7760caf241143f898d4aba7b39f755e for region: 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:54:00,745 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/b2cc6c2d76904b22a26933b36d566d84 for region: 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:54:00,746 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/477cc447542345f2a675a41d3cb656df for region: 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:54:00,747 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/bd22d3c0cbb241fabad930e22fe8b28d for region: 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:54:00,780 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/656e2d9013c348d0a621f1369b58a6b1 for region: 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:54:00,787 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/e715c20fa2c6439d88c153d1aef48552 for region: 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:54:00,791 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/51d64a94e5f64ff684e29f64ff7eaec6 for region: 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:54:00,791 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/286c38e805394011a004447e311f27a7 for region: 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:54:00,791 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/f5d9f2a618634a6e94177f02612aceaf for region: 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:54:00,792 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/2323e300af8249a9bce7f9ed4c4f6fe1 for region: 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:54:00,792 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/bde0a9d3ebfa4e1996e42b844971f197 for region: 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:54:00,792 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region 42fc9cc7beccd63fc2605b5f0fb93a8a Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-05-02 02:54:00,863 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-02 02:54:00,863 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:00,863 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-02 02:54:00,863 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-02 02:54:01,004 INFO [Listener at localhost/37119] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:01,004 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-02 02:54:01,005 INFO [Listener at localhost/37119] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-02 02:54:01,005 INFO [Listener at localhost/37119] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-02 02:54:01,005 ERROR [Listener at localhost/37119] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-02 02:54:01,005 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-02 02:54:01,005 INFO [Listener at localhost/37119] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-02 02:54:01,005 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-02 02:54:01,005 INFO [Listener at localhost/37119] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-02 02:54:01,006 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-02 02:54:01,007 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-05-02 02:54:01,007 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-02 02:54:01,007 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-02 02:54:01,008 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-02 02:54:01,008 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-02 02:54:01,009 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-02 02:54:01,009 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-02 02:54:01,009 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:54:01,010 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=084b1607f7ed8615120a04e4729fbafa, daughterA=ea7683cd3436a62eb16418a3c875d563, daughterB=408e891cab9b919f4038b233ba34779d 2023-05-02 02:54:01,010 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:54:01,010 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:54:01,010 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-02 02:54:01,011 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:54:01,011 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-02 02:54:01,011 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-02 02:54:01,012 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-02 02:54:01,012 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:01,012 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, UNASSIGN 2023-05-02 02:54:01,013 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 42fc9cc7beccd63fc2605b5f0fb93a8a, server=jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:01,013 DEBUG [Listener at localhost/37119] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 held the lock before restarting, call acquireLock to restore it. 2023-05-02 02:54:01,013 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:01,013 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:01,013 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:01,013 DEBUG [Listener at localhost/37119] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-02 02:54:01,013 DEBUG [Listener at localhost/37119] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 42fc9cc7beccd63fc2605b5f0fb93a8a, server=jenkins-hbase4.apache.org,39081,1682996021579 didn't hold the lock before restarting, skip acquiring lock. 2023-05-02 02:54:01,013 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-05-02 02:54:01,014 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-02 02:54:01,015 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,38925,1682996021633, table=hbase:meta, region=1588230740 2023-05-02 02:54:01,016 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682996071016 2023-05-02 02:54:01,017 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:01,017 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(1517): Joining cluster... 2023-05-02 02:54:01,020 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=5c823d6f15ad7b8fe82f9ae4beee6513, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42019,1682996021676, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676, openSeqNum=2 2023-05-02 02:54:01,020 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=42fc9cc7beccd63fc2605b5f0fb93a8a, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,39081,1682996021579, regionLocation=jenkins-hbase4.apache.org,39081,1682996021579, openSeqNum=2 2023-05-02 02:54:01,020 INFO [Listener at localhost/37119] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-02 02:54:01,020 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682996101020 2023-05-02 02:54:01,020 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682996161020 2023-05-02 02:54:01,020 INFO [Listener at localhost/37119] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-05-02 02:54:01,029 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=42fc9cc7beccd63fc2605b5f0fb93a8a, threads=14 2023-05-02 02:54:01,032 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/2323e300af8249a9bce7f9ed4c4f6fe1 for region: 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:54:01,032 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/27e7a682d2094cedbac73b1eaab3471c for region: 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:54:01,032 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/477cc447542345f2a675a41d3cb656df for region: 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:54:01,032 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/51d64a94e5f64ff684e29f64ff7eaec6 for region: 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:54:01,032 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/65211868ead84a0895c2ebf9105167e0 for region: 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:54:01,032 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/656e2d9013c348d0a621f1369b58a6b1 for region: 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:54:01,033 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/f6c6cd94aaf94c469cc58db060f94145 for region: 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:54:01,033 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/286c38e805394011a004447e311f27a7 for region: 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:54:01,033 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/b2cc6c2d76904b22a26933b36d566d84 for region: 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:54:01,034 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/b7760caf241143f898d4aba7b39f755e for region: 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:54:01,034 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/bd22d3c0cbb241fabad930e22fe8b28d for region: 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:54:01,034 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/bde0a9d3ebfa4e1996e42b844971f197 for region: 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:54:01,035 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/f5d9f2a618634a6e94177f02612aceaf for region: 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:54:01,035 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/e715c20fa2c6439d88c153d1aef48552 for region: 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:54:01,145 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-05-02 02:54:01,201 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/477cc447542345f2a675a41d3cb656df for region: 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:54:01,244 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/f6c6cd94aaf94c469cc58db060f94145 for region: 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:54:01,244 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/2323e300af8249a9bce7f9ed4c4f6fe1 for region: 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:54:01,279 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/bde0a9d3ebfa4e1996e42b844971f197 for region: 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:54:01,279 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/51d64a94e5f64ff684e29f64ff7eaec6 for region: 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:54:01,284 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/286c38e805394011a004447e311f27a7 for region: 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:54:01,284 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/27e7a682d2094cedbac73b1eaab3471c for region: 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:54:01,287 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/bd22d3c0cbb241fabad930e22fe8b28d for region: 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:54:01,300 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/b2cc6c2d76904b22a26933b36d566d84 for region: 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:54:01,308 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/65211868ead84a0895c2ebf9105167e0 for region: 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:54:01,324 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/f5d9f2a618634a6e94177f02612aceaf for region: 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:54:01,324 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/656e2d9013c348d0a621f1369b58a6b1 for region: 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:54:01,328 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/e715c20fa2c6439d88c153d1aef48552 for region: 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:54:01,637 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/b7760caf241143f898d4aba7b39f755e for region: 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:54:01,637 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region 42fc9cc7beccd63fc2605b5f0fb93a8a Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-05-02 02:54:01,674 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-02 02:54:01,680 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/dc88a1a6241e1b8594ba9db18769364a/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-05-02 02:54:01,681 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/cd6c9dca49228dd6658df10e124fb142/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-05-02 02:54:01,682 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-02 02:54:01,682 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:01,682 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-02 02:54:01,682 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-02 02:54:01,770 INFO [Listener at localhost/37119] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:01,771 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-02 02:54:01,771 INFO [Listener at localhost/37119] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-02 02:54:01,771 INFO [Listener at localhost/37119] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-02 02:54:01,771 ERROR [Listener at localhost/37119] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-02 02:54:01,771 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-02 02:54:01,771 INFO [Listener at localhost/37119] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-02 02:54:01,771 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-02 02:54:01,771 INFO [Listener at localhost/37119] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-02 02:54:01,772 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-02 02:54:01,773 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-05-02 02:54:01,773 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-02 02:54:01,773 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-02 02:54:01,773 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-02 02:54:01,774 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-02 02:54:01,774 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-02 02:54:01,774 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-02 02:54:01,774 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:54:01,775 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=084b1607f7ed8615120a04e4729fbafa, daughterA=ea7683cd3436a62eb16418a3c875d563, daughterB=408e891cab9b919f4038b233ba34779d 2023-05-02 02:54:01,775 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:54:01,775 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:54:01,775 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-02 02:54:01,775 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:54:01,775 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-02 02:54:01,776 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-02 02:54:01,776 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-02 02:54:01,776 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:01,776 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, UNASSIGN 2023-05-02 02:54:01,777 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 42fc9cc7beccd63fc2605b5f0fb93a8a, server=jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:01,779 DEBUG [Listener at localhost/37119] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 held the lock before restarting, call acquireLock to restore it. 2023-05-02 02:54:01,779 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:01,779 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:01,779 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:01,779 DEBUG [Listener at localhost/37119] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-02 02:54:01,779 DEBUG [Listener at localhost/37119] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 42fc9cc7beccd63fc2605b5f0fb93a8a, server=jenkins-hbase4.apache.org,39081,1682996021579 didn't hold the lock before restarting, skip acquiring lock. 2023-05-02 02:54:01,780 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-05-02 02:54:01,780 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-02 02:54:01,781 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,38925,1682996021633, table=hbase:meta, region=1588230740 2023-05-02 02:54:01,782 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682996071782 2023-05-02 02:54:01,784 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(1517): Joining cluster... 2023-05-02 02:54:01,784 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:01,786 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=5c823d6f15ad7b8fe82f9ae4beee6513, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42019,1682996021676, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676, openSeqNum=2 2023-05-02 02:54:01,786 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=42fc9cc7beccd63fc2605b5f0fb93a8a, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,39081,1682996021579, regionLocation=jenkins-hbase4.apache.org,39081,1682996021579, openSeqNum=2 2023-05-02 02:54:01,786 INFO [Listener at localhost/37119] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-02 02:54:01,786 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682996101786 2023-05-02 02:54:01,786 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682996161786 2023-05-02 02:54:01,786 INFO [Listener at localhost/37119] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-02 02:54:01,789 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-02 02:54:01,792 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-02 02:54:01,792 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:01,793 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-02 02:54:01,793 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-02 02:54:02,037 INFO [Listener at localhost/37119] procedure.MasterProcedureTestingUtility(509): Restart 0 rollback state: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:02,037 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-02 02:54:02,037 INFO [Listener at localhost/37119] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-02 02:54:02,037 INFO [Listener at localhost/37119] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-02 02:54:02,038 ERROR [Listener at localhost/37119] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-02 02:54:02,038 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-02 02:54:02,038 INFO [Listener at localhost/37119] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-02 02:54:02,039 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-02 02:54:02,039 INFO [Listener at localhost/37119] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-02 02:54:02,039 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-02 02:54:02,040 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-05-02 02:54:02,041 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-02 02:54:02,041 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-02 02:54:02,041 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-02 02:54:02,041 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-02 02:54:02,041 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-02 02:54:02,041 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-02 02:54:02,042 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:54:02,042 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=084b1607f7ed8615120a04e4729fbafa, daughterA=ea7683cd3436a62eb16418a3c875d563, daughterB=408e891cab9b919f4038b233ba34779d 2023-05-02 02:54:02,042 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:54:02,042 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:54:02,042 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-02 02:54:02,043 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:54:02,043 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-02 02:54:02,043 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-02 02:54:02,043 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-02 02:54:02,044 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:02,044 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, UNASSIGN 2023-05-02 02:54:02,044 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 42fc9cc7beccd63fc2605b5f0fb93a8a, server=jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:02,044 DEBUG [Listener at localhost/37119] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 held the lock before restarting, call acquireLock to restore it. 2023-05-02 02:54:02,044 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:02,044 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:02,044 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:02,044 DEBUG [Listener at localhost/37119] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-02 02:54:02,044 DEBUG [Listener at localhost/37119] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 42fc9cc7beccd63fc2605b5f0fb93a8a, server=jenkins-hbase4.apache.org,39081,1682996021579 didn't hold the lock before restarting, skip acquiring lock. 2023-05-02 02:54:02,044 DEBUG [Listener at localhost/37119] procedure2.StateMachineProcedure(225): Abort requested for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:02,044 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-05-02 02:54:02,045 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-02 02:54:02,045 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,38925,1682996021633, table=hbase:meta, region=1588230740 2023-05-02 02:54:02,046 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682996072046 2023-05-02 02:54:02,049 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:02,049 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(1517): Joining cluster... 2023-05-02 02:54:02,051 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=5c823d6f15ad7b8fe82f9ae4beee6513, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42019,1682996021676, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676, openSeqNum=2 2023-05-02 02:54:02,051 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=42fc9cc7beccd63fc2605b5f0fb93a8a, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,39081,1682996021579, regionLocation=jenkins-hbase4.apache.org,39081,1682996021579, openSeqNum=2 2023-05-02 02:54:02,051 INFO [Listener at localhost/37119] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-02 02:54:02,051 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682996102051 2023-05-02 02:54:02,051 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682996162051 2023-05-02 02:54:02,051 INFO [Listener at localhost/37119] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-05-02 02:54:02,052 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-02 02:54:02,064 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-02 02:54:02,064 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-05-02 02:54:02,064 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-02 02:54:02,064 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-02 02:54:02,303 INFO [Listener at localhost/37119] procedure.MasterProcedureTestingUtility(509): Restart 1 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:02,303 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-02 02:54:02,303 INFO [Listener at localhost/37119] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-02 02:54:02,303 INFO [Listener at localhost/37119] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-02 02:54:02,303 ERROR [Listener at localhost/37119] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-02 02:54:02,303 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-02 02:54:02,303 INFO [Listener at localhost/37119] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-02 02:54:02,303 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-02 02:54:02,303 INFO [Listener at localhost/37119] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-02 02:54:02,304 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-02 02:54:02,305 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-05-02 02:54:02,305 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-02 02:54:02,305 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-02 02:54:02,306 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-02 02:54:02,306 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-02 02:54:02,306 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-02 02:54:02,306 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-02 02:54:02,306 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:54:02,307 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=084b1607f7ed8615120a04e4729fbafa, daughterA=ea7683cd3436a62eb16418a3c875d563, daughterB=408e891cab9b919f4038b233ba34779d 2023-05-02 02:54:02,307 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:54:02,307 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:54:02,307 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-02 02:54:02,307 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:54:02,308 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-02 02:54:02,308 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-02 02:54:02,308 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-02 02:54:02,309 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:02,309 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, UNASSIGN 2023-05-02 02:54:02,309 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 42fc9cc7beccd63fc2605b5f0fb93a8a, server=jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:02,309 DEBUG [Listener at localhost/37119] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 held the lock before restarting, call acquireLock to restore it. 2023-05-02 02:54:02,309 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:02,309 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:02,309 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:02,309 DEBUG [Listener at localhost/37119] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-02 02:54:02,309 DEBUG [Listener at localhost/37119] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 42fc9cc7beccd63fc2605b5f0fb93a8a, server=jenkins-hbase4.apache.org,39081,1682996021579 didn't hold the lock before restarting, skip acquiring lock. 2023-05-02 02:54:02,309 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-05-02 02:54:02,310 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-02 02:54:02,310 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,38925,1682996021633, table=hbase:meta, region=1588230740 2023-05-02 02:54:02,311 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682996072311 2023-05-02 02:54:02,314 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(1517): Joining cluster... 2023-05-02 02:54:02,314 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:02,316 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=5c823d6f15ad7b8fe82f9ae4beee6513, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42019,1682996021676, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676, openSeqNum=2 2023-05-02 02:54:02,316 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=42fc9cc7beccd63fc2605b5f0fb93a8a, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,39081,1682996021579, regionLocation=jenkins-hbase4.apache.org,39081,1682996021579, openSeqNum=2 2023-05-02 02:54:02,316 INFO [Listener at localhost/37119] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-02 02:54:02,316 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682996102316 2023-05-02 02:54:02,316 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682996162316 2023-05-02 02:54:02,317 INFO [Listener at localhost/37119] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-05-02 02:54:02,317 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-02 02:54:02,319 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-02 02:54:02,319 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-05-02 02:54:02,319 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-02 02:54:02,319 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-02 02:54:02,568 INFO [Listener at localhost/37119] procedure.MasterProcedureTestingUtility(509): Restart 2 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:02,568 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-02 02:54:02,568 INFO [Listener at localhost/37119] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-02 02:54:02,568 INFO [Listener at localhost/37119] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-02 02:54:02,569 ERROR [Listener at localhost/37119] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-02 02:54:02,569 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-02 02:54:02,569 INFO [Listener at localhost/37119] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-02 02:54:02,569 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-02 02:54:02,569 INFO [Listener at localhost/37119] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-02 02:54:02,570 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-02 02:54:02,570 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-05-02 02:54:02,571 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-02 02:54:02,571 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-02 02:54:02,571 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-02 02:54:02,571 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-02 02:54:02,571 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-02 02:54:02,572 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-02 02:54:02,572 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:54:02,572 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=084b1607f7ed8615120a04e4729fbafa, daughterA=ea7683cd3436a62eb16418a3c875d563, daughterB=408e891cab9b919f4038b233ba34779d 2023-05-02 02:54:02,572 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:54:02,572 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:54:02,573 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-02 02:54:02,573 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:54:02,573 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-02 02:54:02,573 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-02 02:54:02,573 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-02 02:54:02,574 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:02,574 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, UNASSIGN 2023-05-02 02:54:02,574 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 42fc9cc7beccd63fc2605b5f0fb93a8a, server=jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:02,574 DEBUG [Listener at localhost/37119] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 held the lock before restarting, call acquireLock to restore it. 2023-05-02 02:54:02,574 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:02,574 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:02,575 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:02,575 DEBUG [Listener at localhost/37119] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-02 02:54:02,575 DEBUG [Listener at localhost/37119] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 42fc9cc7beccd63fc2605b5f0fb93a8a, server=jenkins-hbase4.apache.org,39081,1682996021579 didn't hold the lock before restarting, skip acquiring lock. 2023-05-02 02:54:02,575 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-05-02 02:54:02,575 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-02 02:54:02,576 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,38925,1682996021633, table=hbase:meta, region=1588230740 2023-05-02 02:54:02,576 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682996072576 2023-05-02 02:54:02,577 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(1517): Joining cluster... 2023-05-02 02:54:02,578 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:02,581 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=5c823d6f15ad7b8fe82f9ae4beee6513, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42019,1682996021676, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676, openSeqNum=2 2023-05-02 02:54:02,581 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=42fc9cc7beccd63fc2605b5f0fb93a8a, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,39081,1682996021579, regionLocation=jenkins-hbase4.apache.org,39081,1682996021579, openSeqNum=2 2023-05-02 02:54:02,581 INFO [Listener at localhost/37119] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-02 02:54:02,581 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682996102581 2023-05-02 02:54:02,581 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682996162581 2023-05-02 02:54:02,581 INFO [Listener at localhost/37119] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-05-02 02:54:02,581 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-02 02:54:02,584 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-02 02:54:02,584 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-05-02 02:54:02,584 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-02 02:54:02,584 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-02 02:54:02,831 INFO [Listener at localhost/37119] procedure.MasterProcedureTestingUtility(509): Restart 3 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:02,832 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-02 02:54:02,832 INFO [Listener at localhost/37119] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-02 02:54:02,832 INFO [Listener at localhost/37119] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-02 02:54:02,832 ERROR [Listener at localhost/37119] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-02 02:54:02,832 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-02 02:54:02,832 INFO [Listener at localhost/37119] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-02 02:54:02,832 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-02 02:54:02,832 INFO [Listener at localhost/37119] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-02 02:54:02,833 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-02 02:54:02,834 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-05-02 02:54:02,834 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-02 02:54:02,834 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-02 02:54:02,834 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-02 02:54:02,834 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-02 02:54:02,835 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-02 02:54:02,835 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-02 02:54:02,835 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:54:02,835 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=084b1607f7ed8615120a04e4729fbafa, daughterA=ea7683cd3436a62eb16418a3c875d563, daughterB=408e891cab9b919f4038b233ba34779d 2023-05-02 02:54:02,835 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:54:02,835 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:54:02,836 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-02 02:54:02,836 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:54:02,836 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-02 02:54:02,836 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-02 02:54:02,836 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-02 02:54:02,837 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:02,837 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, UNASSIGN 2023-05-02 02:54:02,837 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 42fc9cc7beccd63fc2605b5f0fb93a8a, server=jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:02,837 DEBUG [Listener at localhost/37119] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 held the lock before restarting, call acquireLock to restore it. 2023-05-02 02:54:02,837 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:02,837 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:02,837 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:02,837 DEBUG [Listener at localhost/37119] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-02 02:54:02,837 DEBUG [Listener at localhost/37119] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 42fc9cc7beccd63fc2605b5f0fb93a8a, server=jenkins-hbase4.apache.org,39081,1682996021579 didn't hold the lock before restarting, skip acquiring lock. 2023-05-02 02:54:02,837 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-05-02 02:54:02,838 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-02 02:54:02,838 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,38925,1682996021633, table=hbase:meta, region=1588230740 2023-05-02 02:54:02,840 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682996072840 2023-05-02 02:54:02,842 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(1517): Joining cluster... 2023-05-02 02:54:02,843 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:02,845 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=5c823d6f15ad7b8fe82f9ae4beee6513, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42019,1682996021676, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676, openSeqNum=2 2023-05-02 02:54:02,845 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=42fc9cc7beccd63fc2605b5f0fb93a8a, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,39081,1682996021579, regionLocation=jenkins-hbase4.apache.org,39081,1682996021579, openSeqNum=2 2023-05-02 02:54:02,845 INFO [Listener at localhost/37119] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-02 02:54:02,845 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682996102845 2023-05-02 02:54:02,845 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682996162845 2023-05-02 02:54:02,845 INFO [Listener at localhost/37119] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-05-02 02:54:02,845 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-02 02:54:02,849 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1054): Stored [pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, ASSIGN] 2023-05-02 02:54:02,849 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-02 02:54:02,849 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-05-02 02:54:02,849 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-02 02:54:02,849 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-02 02:54:03,095 INFO [Listener at localhost/37119] procedure.MasterProcedureTestingUtility(509): Restart 4 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:03,096 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-02 02:54:03,096 INFO [Listener at localhost/37119] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-02 02:54:03,096 INFO [Listener at localhost/37119] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-02 02:54:03,096 ERROR [Listener at localhost/37119] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-02 02:54:03,096 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-02 02:54:03,096 INFO [Listener at localhost/37119] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-02 02:54:03,096 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-02 02:54:03,096 INFO [Listener at localhost/37119] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-02 02:54:03,097 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-02 02:54:03,098 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-05-02 02:54:03,098 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-02 02:54:03,098 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-02 02:54:03,098 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-05-02 02:54:03,098 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-02 02:54:03,099 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-02 02:54:03,099 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-02 02:54:03,099 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-02 02:54:03,099 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:54:03,100 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=084b1607f7ed8615120a04e4729fbafa, daughterA=ea7683cd3436a62eb16418a3c875d563, daughterB=408e891cab9b919f4038b233ba34779d 2023-05-02 02:54:03,100 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:54:03,100 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:54:03,100 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-02 02:54:03,100 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:54:03,100 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-02 02:54:03,101 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-02 02:54:03,101 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-02 02:54:03,102 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:03,102 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, UNASSIGN 2023-05-02 02:54:03,102 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 42fc9cc7beccd63fc2605b5f0fb93a8a, server=jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:03,102 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, ASSIGN 2023-05-02 02:54:03,102 DEBUG [Listener at localhost/37119] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 held the lock before restarting, call acquireLock to restore it. 2023-05-02 02:54:03,102 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:03,102 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:03,102 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:03,102 DEBUG [Listener at localhost/37119] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-02 02:54:03,102 DEBUG [Listener at localhost/37119] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 42fc9cc7beccd63fc2605b5f0fb93a8a, server=jenkins-hbase4.apache.org,39081,1682996021579 didn't hold the lock before restarting, skip acquiring lock. 2023-05-02 02:54:03,102 DEBUG [Listener at localhost/37119] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-02 02:54:03,102 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-05-02 02:54:03,103 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-02 02:54:03,103 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,38925,1682996021633, table=hbase:meta, region=1588230740 2023-05-02 02:54:03,104 INFO [Listener at localhost/37119] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a to restore RIT 2023-05-02 02:54:03,104 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682996073104 2023-05-02 02:54:03,104 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(1517): Joining cluster... 2023-05-02 02:54:03,106 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:03,107 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, ASSIGN 2023-05-02 02:54:03,109 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=5c823d6f15ad7b8fe82f9ae4beee6513, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42019,1682996021676, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676, openSeqNum=2 2023-05-02 02:54:03,109 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=42fc9cc7beccd63fc2605b5f0fb93a8a, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,39081,1682996021579, regionLocation=jenkins-hbase4.apache.org,39081,1682996021579, openSeqNum=2 2023-05-02 02:54:03,109 INFO [Listener at localhost/37119] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-02 02:54:03,109 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682996103109 2023-05-02 02:54:03,109 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682996163109 2023-05-02 02:54:03,109 INFO [Listener at localhost/37119] assignment.AssignmentManager(1536): Joined the cluster in 5 msec 2023-05-02 02:54:03,110 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-02 02:54:03,114 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-02 02:54:03,114 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-05-02 02:54:03,114 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-02 02:54:03,115 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-02 02:54:03,360 INFO [Listener at localhost/37119] procedure.MasterProcedureTestingUtility(509): Restart 5 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:03,360 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-02 02:54:03,360 INFO [Listener at localhost/37119] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-02 02:54:03,360 INFO [Listener at localhost/37119] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-02 02:54:03,361 ERROR [Listener at localhost/37119] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-02 02:54:03,361 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-02 02:54:03,361 INFO [Listener at localhost/37119] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-02 02:54:03,361 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-02 02:54:03,361 INFO [Listener at localhost/37119] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-02 02:54:03,362 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-02 02:54:03,363 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-05-02 02:54:03,363 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-05-02 02:54:03,363 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-02 02:54:03,363 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-02 02:54:03,363 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-02 02:54:03,364 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-02 02:54:03,364 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:54:03,364 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=084b1607f7ed8615120a04e4729fbafa, daughterA=ea7683cd3436a62eb16418a3c875d563, daughterB=408e891cab9b919f4038b233ba34779d 2023-05-02 02:54:03,364 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:54:03,365 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:54:03,365 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-02 02:54:03,365 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:54:03,365 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-02 02:54:03,365 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-02 02:54:03,366 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-02 02:54:03,366 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:03,366 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, ASSIGN 2023-05-02 02:54:03,366 DEBUG [Listener at localhost/37119] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 held the lock before restarting, call acquireLock to restore it. 2023-05-02 02:54:03,367 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:03,367 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:03,367 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:03,367 DEBUG [Listener at localhost/37119] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-02 02:54:03,367 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-05-02 02:54:03,368 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-02 02:54:03,369 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,38925,1682996021633, table=hbase:meta, region=1588230740 2023-05-02 02:54:03,369 INFO [Listener at localhost/37119] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a to restore RIT 2023-05-02 02:54:03,369 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682996073369 2023-05-02 02:54:03,370 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(1517): Joining cluster... 2023-05-02 02:54:03,371 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:03,372 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, ASSIGN 2023-05-02 02:54:03,374 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=5c823d6f15ad7b8fe82f9ae4beee6513, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42019,1682996021676, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676, openSeqNum=2 2023-05-02 02:54:03,374 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=42fc9cc7beccd63fc2605b5f0fb93a8a, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,39081,1682996021579, regionLocation=jenkins-hbase4.apache.org,39081,1682996021579, openSeqNum=2 2023-05-02 02:54:03,374 INFO [Listener at localhost/37119] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-02 02:54:03,374 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682996103374 2023-05-02 02:54:03,374 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682996163374 2023-05-02 02:54:03,374 INFO [Listener at localhost/37119] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-05-02 02:54:03,374 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-02 02:54:03,376 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-02 02:54:03,377 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-05-02 02:54:03,377 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-02 02:54:03,377 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-02 02:54:03,624 INFO [Listener at localhost/37119] procedure.MasterProcedureTestingUtility(509): Restart 6 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:03,625 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-02 02:54:03,625 INFO [Listener at localhost/37119] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-02 02:54:03,625 INFO [Listener at localhost/37119] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-02 02:54:03,626 ERROR [Listener at localhost/37119] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-02 02:54:03,626 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-02 02:54:03,626 INFO [Listener at localhost/37119] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-02 02:54:03,626 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-02 02:54:03,626 INFO [Listener at localhost/37119] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-02 02:54:03,627 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-02 02:54:03,628 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-05-02 02:54:03,628 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-05-02 02:54:03,628 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-02 02:54:03,628 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-02 02:54:03,628 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-02 02:54:03,629 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-02 02:54:03,629 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:54:03,629 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=084b1607f7ed8615120a04e4729fbafa, daughterA=ea7683cd3436a62eb16418a3c875d563, daughterB=408e891cab9b919f4038b233ba34779d 2023-05-02 02:54:03,629 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:54:03,629 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:54:03,630 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-02 02:54:03,630 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:54:03,630 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-02 02:54:03,630 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-02 02:54:03,630 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-02 02:54:03,630 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:03,631 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, ASSIGN 2023-05-02 02:54:03,631 DEBUG [Listener at localhost/37119] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 held the lock before restarting, call acquireLock to restore it. 2023-05-02 02:54:03,631 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:03,631 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:03,631 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:03,631 DEBUG [Listener at localhost/37119] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-02 02:54:03,631 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-05-02 02:54:03,632 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-02 02:54:03,632 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,38925,1682996021633, table=hbase:meta, region=1588230740 2023-05-02 02:54:03,632 INFO [Listener at localhost/37119] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a to restore RIT 2023-05-02 02:54:03,633 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682996073633 2023-05-02 02:54:03,633 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(1517): Joining cluster... 2023-05-02 02:54:03,635 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:03,636 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, ASSIGN 2023-05-02 02:54:03,637 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=5c823d6f15ad7b8fe82f9ae4beee6513, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42019,1682996021676, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676, openSeqNum=2 2023-05-02 02:54:03,638 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=42fc9cc7beccd63fc2605b5f0fb93a8a, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,39081,1682996021579, regionLocation=jenkins-hbase4.apache.org,39081,1682996021579, openSeqNum=2 2023-05-02 02:54:03,638 INFO [Listener at localhost/37119] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-02 02:54:03,638 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682996103638 2023-05-02 02:54:03,638 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682996163638 2023-05-02 02:54:03,638 INFO [Listener at localhost/37119] assignment.AssignmentManager(1536): Joined the cluster in 5 msec 2023-05-02 02:54:03,638 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-02 02:54:03,641 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-02 02:54:03,641 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-05-02 02:54:03,641 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-02 02:54:03,641 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-02 02:54:03,888 INFO [Listener at localhost/37119] procedure.MasterProcedureTestingUtility(509): Restart 7 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:03,888 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-02 02:54:03,888 INFO [Listener at localhost/37119] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-02 02:54:03,888 INFO [Listener at localhost/37119] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-02 02:54:03,889 ERROR [Listener at localhost/37119] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-02 02:54:03,889 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-02 02:54:03,889 INFO [Listener at localhost/37119] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-02 02:54:03,889 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-02 02:54:03,889 INFO [Listener at localhost/37119] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-02 02:54:03,890 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-02 02:54:03,890 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-05-02 02:54:03,890 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-05-02 02:54:03,891 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-02 02:54:03,891 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-02 02:54:03,891 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-02 02:54:03,891 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-02 02:54:03,892 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:54:03,892 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=084b1607f7ed8615120a04e4729fbafa, daughterA=ea7683cd3436a62eb16418a3c875d563, daughterB=408e891cab9b919f4038b233ba34779d 2023-05-02 02:54:03,892 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:54:03,892 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:54:03,892 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-02 02:54:03,892 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:54:03,893 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-02 02:54:03,893 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-02 02:54:03,893 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-02 02:54:03,893 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:03,893 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, ASSIGN 2023-05-02 02:54:03,893 DEBUG [Listener at localhost/37119] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 held the lock before restarting, call acquireLock to restore it. 2023-05-02 02:54:03,893 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:03,893 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:03,894 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:03,894 DEBUG [Listener at localhost/37119] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-02 02:54:03,894 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 3 msec 2023-05-02 02:54:03,894 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-02 02:54:03,895 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,38925,1682996021633, table=hbase:meta, region=1588230740 2023-05-02 02:54:03,895 INFO [Listener at localhost/37119] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a to restore RIT 2023-05-02 02:54:03,895 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682996073895 2023-05-02 02:54:03,897 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(1517): Joining cluster... 2023-05-02 02:54:03,897 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:03,899 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, ASSIGN 2023-05-02 02:54:03,901 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=5c823d6f15ad7b8fe82f9ae4beee6513, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42019,1682996021676, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676, openSeqNum=2 2023-05-02 02:54:03,901 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=42fc9cc7beccd63fc2605b5f0fb93a8a, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,39081,1682996021579, regionLocation=jenkins-hbase4.apache.org,39081,1682996021579, openSeqNum=2 2023-05-02 02:54:03,902 INFO [Listener at localhost/37119] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-02 02:54:03,902 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682996103902 2023-05-02 02:54:03,902 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682996163902 2023-05-02 02:54:03,902 INFO [Listener at localhost/37119] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-05-02 02:54:03,902 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-02 02:54:03,904 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 exec-time=5.2640 sec 2023-05-02 02:54:03,904 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, ASSIGN 2023-05-02 02:54:03,905 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,39081,1682996021579; forceNewPlan=false, retain=true 2023-05-02 02:54:04,055 INFO [jenkins-hbase4:41119] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-05-02 02:54:04,057 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=42fc9cc7beccd63fc2605b5f0fb93a8a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:04,057 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1682996044057"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1682996044057"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682996044057"}]},"ts":"1682996044057"} 2023-05-02 02:54:04,058 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure 42fc9cc7beccd63fc2605b5f0fb93a8a, server=jenkins-hbase4.apache.org,39081,1682996021579}] 2023-05-02 02:54:04,059 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-02 02:54:04,059 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, ASSIGN 2023-05-02 02:54:04,059 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-02 02:54:04,059 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, ASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-02 02:54:04,152 WARN [Listener at localhost/37119] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-02 02:54:04,152 WARN [Listener at localhost/37119] procedure.MasterProcedureTestingUtility(526): ProcedureExecutor not running, may have been stopped by pending procedure due to KillAndToggleBeforeStoreUpdate flag. 2023-05-02 02:54:04,152 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-02 02:54:04,152 INFO [Listener at localhost/37119] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-02 02:54:04,152 INFO [Listener at localhost/37119] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-02 02:54:04,155 ERROR [Listener at localhost/37119] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-02 02:54:04,155 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-02 02:54:04,155 INFO [Listener at localhost/37119] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-02 02:54:04,155 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-02 02:54:04,155 INFO [Listener at localhost/37119] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-02 02:54:04,155 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-02 02:54:04,156 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[0] 2023-05-02 02:54:04,156 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-02 02:54:04,157 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-02 02:54:04,157 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-02 02:54:04,157 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-02 02:54:04,157 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:54:04,157 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=084b1607f7ed8615120a04e4729fbafa, daughterA=ea7683cd3436a62eb16418a3c875d563, daughterB=408e891cab9b919f4038b233ba34779d 2023-05-02 02:54:04,158 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:54:04,158 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:54:04,158 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-02 02:54:04,158 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:54:04,158 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-02 02:54:04,158 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-02 02:54:04,159 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-02 02:54:04,159 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:54:04,159 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, ASSIGN 2023-05-02 02:54:04,159 DEBUG [Listener at localhost/37119] procedure2.Procedure(966): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, ASSIGN held the lock before restarting, call acquireLock to restore it. 2023-05-02 02:54:04,159 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, ASSIGN 2023-05-02 02:54:04,159 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 3 msec 2023-05-02 02:54:04,160 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-02 02:54:04,160 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,38925,1682996021633, table=hbase:meta, region=1588230740 2023-05-02 02:54:04,161 INFO [Listener at localhost/37119] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a to restore RIT 2023-05-02 02:54:04,161 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682996074161 2023-05-02 02:54:04,162 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, ASSIGN 2023-05-02 02:54:04,162 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(1517): Joining cluster... 2023-05-02 02:54:04,164 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=5c823d6f15ad7b8fe82f9ae4beee6513, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42019,1682996021676, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676, openSeqNum=2 2023-05-02 02:54:04,165 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=42fc9cc7beccd63fc2605b5f0fb93a8a, regionState=OPENING, lastHost=jenkins-hbase4.apache.org,39081,1682996021579, regionLocation=jenkins-hbase4.apache.org,39081,1682996021579, openSeqNum=2 2023-05-02 02:54:04,165 INFO [Listener at localhost/37119] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-02 02:54:04,165 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682996104165 2023-05-02 02:54:04,165 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682996164165 2023-05-02 02:54:04,165 INFO [Listener at localhost/37119] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-05-02 02:54:04,165 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=42fc9cc7beccd63fc2605b5f0fb93a8a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:04,165 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1682996044165"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1682996044165"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682996044165"}]},"ts":"1682996044165"} 2023-05-02 02:54:04,167 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure 42fc9cc7beccd63fc2605b5f0fb93a8a, server=jenkins-hbase4.apache.org,39081,1682996021579}] 2023-05-02 02:54:04,323 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a. 2023-05-02 02:54:04,324 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 42fc9cc7beccd63fc2605b5f0fb93a8a, NAME => 'testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a.', STARTKEY => '', ENDKEY => ''} 2023-05-02 02:54:04,324 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-02 02:54:04,324 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:54:04,324 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-02 02:54:04,324 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:54:04,324 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:54:04,325 INFO [StoreOpener-42fc9cc7beccd63fc2605b5f0fb93a8a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:54:04,326 DEBUG [StoreOpener-42fc9cc7beccd63fc2605b5f0fb93a8a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1 2023-05-02 02:54:04,326 DEBUG [StoreOpener-42fc9cc7beccd63fc2605b5f0fb93a8a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1 2023-05-02 02:54:04,327 INFO [StoreOpener-42fc9cc7beccd63fc2605b5f0fb93a8a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 42fc9cc7beccd63fc2605b5f0fb93a8a columnFamilyName cf1 2023-05-02 02:54:04,334 DEBUG [StoreOpener-42fc9cc7beccd63fc2605b5f0fb93a8a-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/286c38e805394011a004447e311f27a7 2023-05-02 02:54:04,338 DEBUG [StoreOpener-42fc9cc7beccd63fc2605b5f0fb93a8a-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/b2cc6c2d76904b22a26933b36d566d84 2023-05-02 02:54:04,341 DEBUG [StoreOpener-42fc9cc7beccd63fc2605b5f0fb93a8a-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/b7760caf241143f898d4aba7b39f755e 2023-05-02 02:54:04,345 DEBUG [StoreOpener-42fc9cc7beccd63fc2605b5f0fb93a8a-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/bd22d3c0cbb241fabad930e22fe8b28d 2023-05-02 02:54:04,352 DEBUG [StoreOpener-42fc9cc7beccd63fc2605b5f0fb93a8a-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/bde0a9d3ebfa4e1996e42b844971f197 2023-05-02 02:54:04,360 DEBUG [StoreOpener-42fc9cc7beccd63fc2605b5f0fb93a8a-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/e715c20fa2c6439d88c153d1aef48552 2023-05-02 02:54:04,364 DEBUG [StoreOpener-42fc9cc7beccd63fc2605b5f0fb93a8a-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/f5d9f2a618634a6e94177f02612aceaf 2023-05-02 02:54:04,364 INFO [StoreOpener-42fc9cc7beccd63fc2605b5f0fb93a8a-1] regionserver.HStore(310): Store=42fc9cc7beccd63fc2605b5f0fb93a8a/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-02 02:54:04,364 INFO [StoreOpener-42fc9cc7beccd63fc2605b5f0fb93a8a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:54:04,365 DEBUG [StoreOpener-42fc9cc7beccd63fc2605b5f0fb93a8a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2 2023-05-02 02:54:04,365 DEBUG [StoreOpener-42fc9cc7beccd63fc2605b5f0fb93a8a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2 2023-05-02 02:54:04,366 INFO [StoreOpener-42fc9cc7beccd63fc2605b5f0fb93a8a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 42fc9cc7beccd63fc2605b5f0fb93a8a columnFamilyName cf2 2023-05-02 02:54:04,377 DEBUG [StoreOpener-42fc9cc7beccd63fc2605b5f0fb93a8a-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/2323e300af8249a9bce7f9ed4c4f6fe1 2023-05-02 02:54:04,380 DEBUG [StoreOpener-42fc9cc7beccd63fc2605b5f0fb93a8a-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/27e7a682d2094cedbac73b1eaab3471c 2023-05-02 02:54:04,386 DEBUG [StoreOpener-42fc9cc7beccd63fc2605b5f0fb93a8a-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/477cc447542345f2a675a41d3cb656df 2023-05-02 02:54:04,389 DEBUG [StoreOpener-42fc9cc7beccd63fc2605b5f0fb93a8a-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/51d64a94e5f64ff684e29f64ff7eaec6 2023-05-02 02:54:04,393 DEBUG [StoreOpener-42fc9cc7beccd63fc2605b5f0fb93a8a-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/65211868ead84a0895c2ebf9105167e0 2023-05-02 02:54:04,400 DEBUG [StoreOpener-42fc9cc7beccd63fc2605b5f0fb93a8a-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/656e2d9013c348d0a621f1369b58a6b1 2023-05-02 02:54:04,403 DEBUG [StoreOpener-42fc9cc7beccd63fc2605b5f0fb93a8a-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/f6c6cd94aaf94c469cc58db060f94145 2023-05-02 02:54:04,404 INFO [StoreOpener-42fc9cc7beccd63fc2605b5f0fb93a8a-1] regionserver.HStore(310): Store=42fc9cc7beccd63fc2605b5f0fb93a8a/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-02 02:54:04,404 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:54:04,405 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:54:04,407 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(602): Cleaned up old failed split transaction detritus: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/.splits 2023-05-02 02:54:04,408 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-02 02:54:04,409 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:54:04,410 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 42fc9cc7beccd63fc2605b5f0fb93a8a; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63454775, jitterRate=-0.05445016920566559}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-02 02:54:04,410 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 42fc9cc7beccd63fc2605b5f0fb93a8a: 2023-05-02 02:54:04,411 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a., pid=48, masterSystemTime=1682996044319 2023-05-02 02:54:04,411 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-02 02:54:04,412 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-05-02 02:54:04,412 DEBUG [RS:0;jenkins-hbase4:39081-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-02 02:54:04,412 DEBUG [RS:0;jenkins-hbase4:39081-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-02 02:54:04,413 DEBUG [RS:0;jenkins-hbase4:39081-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-05-02 02:54:04,413 DEBUG [RS:0;jenkins-hbase4:39081-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-05-02 02:54:04,413 DEBUG [RS:0;jenkins-hbase4:39081-shortCompactions-0] regionserver.HStore(1912): 42fc9cc7beccd63fc2605b5f0fb93a8a/cf1 is initiating minor compaction (all files) 2023-05-02 02:54:04,413 DEBUG [RS:0;jenkins-hbase4:39081-longCompactions-0] regionserver.HStore(1912): 42fc9cc7beccd63fc2605b5f0fb93a8a/cf2 is initiating minor compaction (all files) 2023-05-02 02:54:04,413 INFO [RS:0;jenkins-hbase4:39081-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 42fc9cc7beccd63fc2605b5f0fb93a8a/cf1 in testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a. 2023-05-02 02:54:04,413 INFO [RS:0;jenkins-hbase4:39081-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 42fc9cc7beccd63fc2605b5f0fb93a8a/cf2 in testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a. 2023-05-02 02:54:04,414 INFO [RS:0;jenkins-hbase4:39081-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/bde0a9d3ebfa4e1996e42b844971f197, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/286c38e805394011a004447e311f27a7, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/f5d9f2a618634a6e94177f02612aceaf, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/b2cc6c2d76904b22a26933b36d566d84, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/b7760caf241143f898d4aba7b39f755e, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/bd22d3c0cbb241fabad930e22fe8b28d, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/e715c20fa2c6439d88c153d1aef48552] into tmpdir=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/.tmp, totalSize=34.9 K 2023-05-02 02:54:04,414 INFO [RS:0;jenkins-hbase4:39081-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/65211868ead84a0895c2ebf9105167e0, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/27e7a682d2094cedbac73b1eaab3471c, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/2323e300af8249a9bce7f9ed4c4f6fe1, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/656e2d9013c348d0a621f1369b58a6b1, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/f6c6cd94aaf94c469cc58db060f94145, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/477cc447542345f2a675a41d3cb656df, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/51d64a94e5f64ff684e29f64ff7eaec6] into tmpdir=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/.tmp, totalSize=34.9 K 2023-05-02 02:54:04,415 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a. 2023-05-02 02:54:04,415 DEBUG [RS:0;jenkins-hbase4:39081-shortCompactions-0] compactions.Compactor(207): Compacting bde0a9d3ebfa4e1996e42b844971f197, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1682996037174 2023-05-02 02:54:04,415 DEBUG [RS:0;jenkins-hbase4:39081-longCompactions-0] compactions.Compactor(207): Compacting 65211868ead84a0895c2ebf9105167e0, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1682996037174 2023-05-02 02:54:04,415 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a. 2023-05-02 02:54:04,415 DEBUG [RS:0;jenkins-hbase4:39081-shortCompactions-0] compactions.Compactor(207): Compacting 286c38e805394011a004447e311f27a7, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1682996037388 2023-05-02 02:54:04,416 DEBUG [RS:0;jenkins-hbase4:39081-longCompactions-0] compactions.Compactor(207): Compacting 27e7a682d2094cedbac73b1eaab3471c, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1682996037388 2023-05-02 02:54:04,416 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=42fc9cc7beccd63fc2605b5f0fb93a8a, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:04,416 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1682996044415"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1682996044415"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1682996044415"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682996044415"}]},"ts":"1682996044415"} 2023-05-02 02:54:04,416 DEBUG [RS:0;jenkins-hbase4:39081-shortCompactions-0] compactions.Compactor(207): Compacting f5d9f2a618634a6e94177f02612aceaf, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1682996037599 2023-05-02 02:54:04,416 DEBUG [RS:0;jenkins-hbase4:39081-longCompactions-0] compactions.Compactor(207): Compacting 2323e300af8249a9bce7f9ed4c4f6fe1, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1682996037599 2023-05-02 02:54:04,416 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(335): procId=44 exception: org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested 2023-05-02 02:54:04,416 DEBUG [RS:0;jenkins-hbase4:39081-shortCompactions-0] compactions.Compactor(207): Compacting b2cc6c2d76904b22a26933b36d566d84, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1682996037795 2023-05-02 02:54:04,417 DEBUG [RS:0;jenkins-hbase4:39081-longCompactions-0] compactions.Compactor(207): Compacting 656e2d9013c348d0a621f1369b58a6b1, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1682996037795 2023-05-02 02:54:04,417 DEBUG [RS:0;jenkins-hbase4:39081-shortCompactions-0] compactions.Compactor(207): Compacting b7760caf241143f898d4aba7b39f755e, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1682996037987 2023-05-02 02:54:04,417 DEBUG [RS:0;jenkins-hbase4:39081-longCompactions-0] compactions.Compactor(207): Compacting f6c6cd94aaf94c469cc58db060f94145, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1682996037987 2023-05-02 02:54:04,417 DEBUG [RS:0;jenkins-hbase4:39081-shortCompactions-0] compactions.Compactor(207): Compacting bd22d3c0cbb241fabad930e22fe8b28d, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1682996038185 2023-05-02 02:54:04,418 DEBUG [RS:0;jenkins-hbase4:39081-longCompactions-0] compactions.Compactor(207): Compacting 477cc447542345f2a675a41d3cb656df, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1682996038185 2023-05-02 02:54:04,418 DEBUG [RS:0;jenkins-hbase4:39081-shortCompactions-0] compactions.Compactor(207): Compacting e715c20fa2c6439d88c153d1aef48552, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1682996038377 2023-05-02 02:54:04,418 DEBUG [RS:0;jenkins-hbase4:39081-longCompactions-0] compactions.Compactor(207): Compacting 51d64a94e5f64ff684e29f64ff7eaec6, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1682996038377 2023-05-02 02:54:04,419 DEBUG [Listener at localhost/37119] hbase.HBaseTestingUtility(3430): Waiting until all regions of table testRollbackAndDoubleExecution get assigned. Timeout = 1500ms 2023-05-02 02:54:04,421 INFO [Listener at localhost/37119] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-05-02 02:54:04,424 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=48, resume processing ppid=47 2023-05-02 02:54:04,424 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=48, ppid=47, state=SUCCESS; OpenRegionProcedure 42fc9cc7beccd63fc2605b5f0fb93a8a, server=jenkins-hbase4.apache.org,39081,1682996021579 in 255 msec 2023-05-02 02:54:04,426 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, ASSIGN in 1.5760 sec 2023-05-02 02:54:04,430 INFO [Listener at localhost/37119] hbase.HBaseTestingUtility(3484): All regions for table testRollbackAndDoubleExecution assigned to meta. Checking AM states. 2023-05-02 02:54:04,431 INFO [Listener at localhost/37119] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-05-02 02:54:04,431 INFO [Listener at localhost/37119] hbase.HBaseTestingUtility(3504): All regions for table testRollbackAndDoubleExecution assigned. 2023-05-02 02:54:04,462 INFO [RS:0;jenkins-hbase4:39081-longCompactions-0] throttle.PressureAwareThroughputController(145): 42fc9cc7beccd63fc2605b5f0fb93a8a#cf2#compaction#50 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-02 02:54:04,462 INFO [RS:0;jenkins-hbase4:39081-shortCompactions-0] throttle.PressureAwareThroughputController(145): 42fc9cc7beccd63fc2605b5f0fb93a8a#cf1#compaction#51 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-02 02:54:04,504 DEBUG [RS:0;jenkins-hbase4:39081-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/.tmp/cf1/1359a7f7ea904e988c91644a53ec49fb as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/1359a7f7ea904e988c91644a53ec49fb 2023-05-02 02:54:04,508 DEBUG [RS:0;jenkins-hbase4:39081-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/.tmp/cf2/20f3f874ebd54810ade2a76ab0a8f8ff as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/20f3f874ebd54810ade2a76ab0a8f8ff 2023-05-02 02:54:04,511 INFO [RS:0;jenkins-hbase4:39081-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 42fc9cc7beccd63fc2605b5f0fb93a8a/cf1 of 42fc9cc7beccd63fc2605b5f0fb93a8a into 1359a7f7ea904e988c91644a53ec49fb(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-02 02:54:04,511 DEBUG [RS:0;jenkins-hbase4:39081-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 42fc9cc7beccd63fc2605b5f0fb93a8a: 2023-05-02 02:54:04,511 INFO [RS:0;jenkins-hbase4:39081-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a., storeName=42fc9cc7beccd63fc2605b5f0fb93a8a/cf1, priority=9, startTime=1682996044411; duration=0sec 2023-05-02 02:54:04,513 DEBUG [RS:0;jenkins-hbase4:39081-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-02 02:54:04,518 INFO [RS:0;jenkins-hbase4:39081-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 42fc9cc7beccd63fc2605b5f0fb93a8a/cf2 of 42fc9cc7beccd63fc2605b5f0fb93a8a into 20f3f874ebd54810ade2a76ab0a8f8ff(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-02 02:54:04,518 DEBUG [RS:0;jenkins-hbase4:39081-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 42fc9cc7beccd63fc2605b5f0fb93a8a: 2023-05-02 02:54:04,519 INFO [RS:0;jenkins-hbase4:39081-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a., storeName=42fc9cc7beccd63fc2605b5f0fb93a8a/cf2, priority=9, startTime=1682996044411; duration=0sec 2023-05-02 02:54:04,519 DEBUG [RS:0;jenkins-hbase4:39081-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-02 02:54:04,524 WARN [Listener at localhost/37119] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-02 02:54:04,526 INFO [Listener at localhost/37119] client.HBaseAdmin$15(890): Started disable of testRollbackAndDoubleExecution 2023-05-02 02:54:04,527 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testRollbackAndDoubleExecution 2023-05-02 02:54:04,528 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure2.ProcedureExecutor(1029): Stored pid=49, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-05-02 02:54:04,530 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-05-02 02:54:04,531 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682996044531"}]},"ts":"1682996044531"} 2023-05-02 02:54:04,533 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLING in hbase:meta 2023-05-02 02:54:04,566 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackAndDoubleExecution to state=DISABLING 2023-05-02 02:54:04,567 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, UNASSIGN}] 2023-05-02 02:54:04,567 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, UNASSIGN 2023-05-02 02:54:04,568 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=42fc9cc7beccd63fc2605b5f0fb93a8a, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:04,568 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1682996044568"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1682996044568"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682996044568"}]},"ts":"1682996044568"} 2023-05-02 02:54:04,570 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=51, ppid=50, state=RUNNABLE; CloseRegionProcedure 42fc9cc7beccd63fc2605b5f0fb93a8a, server=jenkins-hbase4.apache.org,39081,1682996021579}] 2023-05-02 02:54:04,632 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-05-02 02:54:04,722 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:54:04,722 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 42fc9cc7beccd63fc2605b5f0fb93a8a, disabling compactions & flushes 2023-05-02 02:54:04,722 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a. 2023-05-02 02:54:04,722 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a. 2023-05-02 02:54:04,722 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a. after waiting 0 ms 2023-05-02 02:54:04,722 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a. 2023-05-02 02:54:04,723 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/bde0a9d3ebfa4e1996e42b844971f197, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/286c38e805394011a004447e311f27a7, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/f5d9f2a618634a6e94177f02612aceaf, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/b2cc6c2d76904b22a26933b36d566d84, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/b7760caf241143f898d4aba7b39f755e, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/bd22d3c0cbb241fabad930e22fe8b28d, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/e715c20fa2c6439d88c153d1aef48552] to archive 2023-05-02 02:54:04,724 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-02 02:54:04,726 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/bde0a9d3ebfa4e1996e42b844971f197 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/bde0a9d3ebfa4e1996e42b844971f197 2023-05-02 02:54:04,727 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/286c38e805394011a004447e311f27a7 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/286c38e805394011a004447e311f27a7 2023-05-02 02:54:04,728 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/f5d9f2a618634a6e94177f02612aceaf to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/f5d9f2a618634a6e94177f02612aceaf 2023-05-02 02:54:04,729 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/b2cc6c2d76904b22a26933b36d566d84 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/b2cc6c2d76904b22a26933b36d566d84 2023-05-02 02:54:04,730 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/b7760caf241143f898d4aba7b39f755e to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/b7760caf241143f898d4aba7b39f755e 2023-05-02 02:54:04,732 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/bd22d3c0cbb241fabad930e22fe8b28d to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/bd22d3c0cbb241fabad930e22fe8b28d 2023-05-02 02:54:04,733 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/e715c20fa2c6439d88c153d1aef48552 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/e715c20fa2c6439d88c153d1aef48552 2023-05-02 02:54:04,739 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/65211868ead84a0895c2ebf9105167e0, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/27e7a682d2094cedbac73b1eaab3471c, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/2323e300af8249a9bce7f9ed4c4f6fe1, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/656e2d9013c348d0a621f1369b58a6b1, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/f6c6cd94aaf94c469cc58db060f94145, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/477cc447542345f2a675a41d3cb656df, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/51d64a94e5f64ff684e29f64ff7eaec6] to archive 2023-05-02 02:54:04,740 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-02 02:54:04,742 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/65211868ead84a0895c2ebf9105167e0 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/65211868ead84a0895c2ebf9105167e0 2023-05-02 02:54:04,743 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/27e7a682d2094cedbac73b1eaab3471c to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/27e7a682d2094cedbac73b1eaab3471c 2023-05-02 02:54:04,744 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/2323e300af8249a9bce7f9ed4c4f6fe1 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/2323e300af8249a9bce7f9ed4c4f6fe1 2023-05-02 02:54:04,746 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/656e2d9013c348d0a621f1369b58a6b1 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/656e2d9013c348d0a621f1369b58a6b1 2023-05-02 02:54:04,747 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/f6c6cd94aaf94c469cc58db060f94145 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/f6c6cd94aaf94c469cc58db060f94145 2023-05-02 02:54:04,748 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/477cc447542345f2a675a41d3cb656df to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/477cc447542345f2a675a41d3cb656df 2023-05-02 02:54:04,749 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/51d64a94e5f64ff684e29f64ff7eaec6 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/51d64a94e5f64ff684e29f64ff7eaec6 2023-05-02 02:54:04,753 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-05-02 02:54:04,753 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-02 02:54:04,754 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a. 2023-05-02 02:54:04,754 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 42fc9cc7beccd63fc2605b5f0fb93a8a: 2023-05-02 02:54:04,755 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:54:04,756 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=42fc9cc7beccd63fc2605b5f0fb93a8a, regionState=CLOSED 2023-05-02 02:54:04,756 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1682996044755"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682996044755"}]},"ts":"1682996044755"} 2023-05-02 02:54:04,759 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=51, resume processing ppid=50 2023-05-02 02:54:04,759 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=51, ppid=50, state=SUCCESS; CloseRegionProcedure 42fc9cc7beccd63fc2605b5f0fb93a8a, server=jenkins-hbase4.apache.org,39081,1682996021579 in 187 msec 2023-05-02 02:54:04,760 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=50, resume processing ppid=49 2023-05-02 02:54:04,760 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=50, ppid=49, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, UNASSIGN in 192 msec 2023-05-02 02:54:04,761 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682996044761"}]},"ts":"1682996044761"} 2023-05-02 02:54:04,762 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLED in hbase:meta 2023-05-02 02:54:04,763 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackAndDoubleExecution to state=DISABLED 2023-05-02 02:54:04,765 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution in 237 msec 2023-05-02 02:54:04,833 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-05-02 02:54:04,833 INFO [Listener at localhost/37119] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackAndDoubleExecution, procId: 49 completed 2023-05-02 02:54:04,834 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testRollbackAndDoubleExecution 2023-05-02 02:54:04,835 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure2.ProcedureExecutor(1029): Stored pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-05-02 02:54:04,837 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-05-02 02:54:04,838 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=52, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-05-02 02:54:04,838 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1227): Checking to see if procedure is done pid=52 2023-05-02 02:54:04,841 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:54:04,843 DEBUG [HFileArchiver-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1, FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2, FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/recovered.edits] 2023-05-02 02:54:04,845 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/1359a7f7ea904e988c91644a53ec49fb to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf1/1359a7f7ea904e988c91644a53ec49fb 2023-05-02 02:54:04,847 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/20f3f874ebd54810ade2a76ab0a8f8ff to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/cf2/20f3f874ebd54810ade2a76ab0a8f8ff 2023-05-02 02:54:04,850 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/recovered.edits/90.seqid to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a/recovered.edits/90.seqid 2023-05-02 02:54:04,850 DEBUG [HFileArchiver-5] backup.HFileArchiver(596): Deleted hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRollbackAndDoubleExecution/42fc9cc7beccd63fc2605b5f0fb93a8a 2023-05-02 02:54:04,850 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-05-02 02:54:04,854 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=52, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-05-02 02:54:04,855 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackAndDoubleExecution from hbase:meta 2023-05-02 02:54:04,857 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackAndDoubleExecution' descriptor. 2023-05-02 02:54:04,858 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=52, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-05-02 02:54:04,858 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackAndDoubleExecution' from region states. 2023-05-02 02:54:04,858 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1682996044858"}]},"ts":"9223372036854775807"} 2023-05-02 02:54:04,860 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-05-02 02:54:04,860 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 42fc9cc7beccd63fc2605b5f0fb93a8a, NAME => 'testRollbackAndDoubleExecution,,1682996036665.42fc9cc7beccd63fc2605b5f0fb93a8a.', STARTKEY => '', ENDKEY => ''}] 2023-05-02 02:54:04,860 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackAndDoubleExecution' as deleted. 2023-05-02 02:54:04,860 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1682996044860"}]},"ts":"9223372036854775807"} 2023-05-02 02:54:04,862 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackAndDoubleExecution state from META 2023-05-02 02:54:04,863 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=52, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-05-02 02:54:04,864 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution in 29 msec 2023-05-02 02:54:04,939 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1227): Checking to see if procedure is done pid=52 2023-05-02 02:54:04,940 INFO [Listener at localhost/37119] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackAndDoubleExecution, procId: 52 completed 2023-05-02 02:54:04,957 INFO [Listener at localhost/37119] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=402 (was 399) Potentially hanging thread: RS-EventLoopGroup-3-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1250645491_15 at /127.0.0.1:38792 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2c67b29-shared-pool-9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1250645491_15 at /127.0.0.1:40234 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1250645491_15 at /127.0.0.1:40268 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1250645491_15 at /127.0.0.1:38826 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1250645491_15 at /127.0.0.1:59662 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e48d2209-986f-dd5e-f2e4-019027d7bcac/cluster_9b21c39f-0840-34d4-ae73-45b092a6b580/dfs/data/data3/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1250645491_15 at /127.0.0.1:38852 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1250645491_15 at /127.0.0.1:40260 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1250645491_15 at /127.0.0.1:40218 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1250645491_15 at /127.0.0.1:40250 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1250645491_15 at /127.0.0.1:59676 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1250645491_15 at /127.0.0.1:38812 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e48d2209-986f-dd5e-f2e4-019027d7bcac/cluster_9b21c39f-0840-34d4-ae73-45b092a6b580/dfs/data/data4/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1250645491_15 at /127.0.0.1:38832 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1250645491_15 at /127.0.0.1:59696 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1250645491_15 at /127.0.0.1:59682 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2c67b29-shared-pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:0;jenkins-hbase4:39081-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1250645491_15 at /127.0.0.1:38840 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1250645491_15 at /127.0.0.1:38796 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=867 (was 848) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=268 (was 289), ProcessCount=170 (was 170), AvailableMemoryMB=13854 (was 13966) 2023-05-02 02:54:04,972 INFO [Listener at localhost/37119] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=402, OpenFileDescriptor=867, MaxFileDescriptor=60000, SystemLoadAverage=268, ProcessCount=170, AvailableMemoryMB=13853 2023-05-02 02:54:04,972 WARN [Listener at localhost/37119] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-02 02:54:04,975 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-05-02 02:54:04,975 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-05-02 02:54:04,977 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(1029): Stored pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-02 02:54:04,979 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_PRE_OPERATION 2023-05-02 02:54:04,980 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-05-02 02:54:04,982 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionNoStoreFile/afa57eddfa2d26bf35b12dc022351e78 2023-05-02 02:54:04,982 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionNoStoreFile/afa57eddfa2d26bf35b12dc022351e78 empty. 2023-05-02 02:54:04,983 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionNoStoreFile/afa57eddfa2d26bf35b12dc022351e78 2023-05-02 02:54:04,983 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-05-02 02:54:04,993 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionNoStoreFile/.tabledesc/.tableinfo.0000000001 2023-05-02 02:54:04,994 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(7675): creating {ENCODED => afa57eddfa2d26bf35b12dc022351e78, NAME => 'testSplitTableRegionNoStoreFile,,1682996044976.afa57eddfa2d26bf35b12dc022351e78.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionNoStoreFile', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp 2023-05-02 02:54:05,008 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegionNoStoreFile,,1682996044976.afa57eddfa2d26bf35b12dc022351e78.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-02 02:54:05,008 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1604): Closing afa57eddfa2d26bf35b12dc022351e78, disabling compactions & flushes 2023-05-02 02:54:05,008 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegionNoStoreFile,,1682996044976.afa57eddfa2d26bf35b12dc022351e78. 2023-05-02 02:54:05,008 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1682996044976.afa57eddfa2d26bf35b12dc022351e78. 2023-05-02 02:54:05,008 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionNoStoreFile,,1682996044976.afa57eddfa2d26bf35b12dc022351e78. after waiting 0 ms 2023-05-02 02:54:05,008 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionNoStoreFile,,1682996044976.afa57eddfa2d26bf35b12dc022351e78. 2023-05-02 02:54:05,008 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegionNoStoreFile,,1682996044976.afa57eddfa2d26bf35b12dc022351e78. 2023-05-02 02:54:05,008 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1558): Region close journal for afa57eddfa2d26bf35b12dc022351e78: 2023-05-02 02:54:05,011 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ADD_TO_META 2023-05-02 02:54:05,012 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1682996044976.afa57eddfa2d26bf35b12dc022351e78.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1682996045012"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682996045012"}]},"ts":"1682996045012"} 2023-05-02 02:54:05,014 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-05-02 02:54:05,015 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-05-02 02:54:05,015 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682996045015"}]},"ts":"1682996045015"} 2023-05-02 02:54:05,016 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLING in hbase:meta 2023-05-02 02:54:05,019 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-05-02 02:54:05,020 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-05-02 02:54:05,020 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-05-02 02:54:05,020 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-05-02 02:54:05,020 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-05-02 02:54:05,021 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=afa57eddfa2d26bf35b12dc022351e78, ASSIGN}] 2023-05-02 02:54:05,022 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=afa57eddfa2d26bf35b12dc022351e78, ASSIGN 2023-05-02 02:54:05,023 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=afa57eddfa2d26bf35b12dc022351e78, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42019,1682996021676; forceNewPlan=false, retain=false 2023-05-02 02:54:05,173 INFO [jenkins-hbase4:41119] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-05-02 02:54:05,174 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=afa57eddfa2d26bf35b12dc022351e78, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:05,174 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1682996044976.afa57eddfa2d26bf35b12dc022351e78.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1682996045174"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1682996045174"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682996045174"}]},"ts":"1682996045174"} 2023-05-02 02:54:05,176 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=55, ppid=54, state=RUNNABLE; OpenRegionProcedure afa57eddfa2d26bf35b12dc022351e78, server=jenkins-hbase4.apache.org,42019,1682996021676}] 2023-05-02 02:54:05,331 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionNoStoreFile,,1682996044976.afa57eddfa2d26bf35b12dc022351e78. 2023-05-02 02:54:05,331 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => afa57eddfa2d26bf35b12dc022351e78, NAME => 'testSplitTableRegionNoStoreFile,,1682996044976.afa57eddfa2d26bf35b12dc022351e78.', STARTKEY => '', ENDKEY => ''} 2023-05-02 02:54:05,332 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-02 02:54:05,332 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile afa57eddfa2d26bf35b12dc022351e78 2023-05-02 02:54:05,332 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionNoStoreFile,,1682996044976.afa57eddfa2d26bf35b12dc022351e78.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-02 02:54:05,332 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for afa57eddfa2d26bf35b12dc022351e78 2023-05-02 02:54:05,332 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for afa57eddfa2d26bf35b12dc022351e78 2023-05-02 02:54:05,333 INFO [StoreOpener-afa57eddfa2d26bf35b12dc022351e78-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region afa57eddfa2d26bf35b12dc022351e78 2023-05-02 02:54:05,335 DEBUG [StoreOpener-afa57eddfa2d26bf35b12dc022351e78-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionNoStoreFile/afa57eddfa2d26bf35b12dc022351e78/cf1 2023-05-02 02:54:05,335 DEBUG [StoreOpener-afa57eddfa2d26bf35b12dc022351e78-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionNoStoreFile/afa57eddfa2d26bf35b12dc022351e78/cf1 2023-05-02 02:54:05,335 INFO [StoreOpener-afa57eddfa2d26bf35b12dc022351e78-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region afa57eddfa2d26bf35b12dc022351e78 columnFamilyName cf1 2023-05-02 02:54:05,336 INFO [StoreOpener-afa57eddfa2d26bf35b12dc022351e78-1] regionserver.HStore(310): Store=afa57eddfa2d26bf35b12dc022351e78/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-02 02:54:05,336 INFO [StoreOpener-afa57eddfa2d26bf35b12dc022351e78-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region afa57eddfa2d26bf35b12dc022351e78 2023-05-02 02:54:05,337 DEBUG [StoreOpener-afa57eddfa2d26bf35b12dc022351e78-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionNoStoreFile/afa57eddfa2d26bf35b12dc022351e78/cf2 2023-05-02 02:54:05,337 DEBUG [StoreOpener-afa57eddfa2d26bf35b12dc022351e78-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionNoStoreFile/afa57eddfa2d26bf35b12dc022351e78/cf2 2023-05-02 02:54:05,338 INFO [StoreOpener-afa57eddfa2d26bf35b12dc022351e78-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region afa57eddfa2d26bf35b12dc022351e78 columnFamilyName cf2 2023-05-02 02:54:05,338 INFO [StoreOpener-afa57eddfa2d26bf35b12dc022351e78-1] regionserver.HStore(310): Store=afa57eddfa2d26bf35b12dc022351e78/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-02 02:54:05,339 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionNoStoreFile/afa57eddfa2d26bf35b12dc022351e78 2023-05-02 02:54:05,339 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionNoStoreFile/afa57eddfa2d26bf35b12dc022351e78 2023-05-02 02:54:05,341 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-02 02:54:05,343 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for afa57eddfa2d26bf35b12dc022351e78 2023-05-02 02:54:05,345 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionNoStoreFile/afa57eddfa2d26bf35b12dc022351e78/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-05-02 02:54:05,346 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened afa57eddfa2d26bf35b12dc022351e78; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73863554, jitterRate=0.10065272450447083}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-02 02:54:05,346 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for afa57eddfa2d26bf35b12dc022351e78: 2023-05-02 02:54:05,347 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1682996044976.afa57eddfa2d26bf35b12dc022351e78., pid=55, masterSystemTime=1682996045328 2023-05-02 02:54:05,349 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1682996044976.afa57eddfa2d26bf35b12dc022351e78. 2023-05-02 02:54:05,349 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionNoStoreFile,,1682996044976.afa57eddfa2d26bf35b12dc022351e78. 2023-05-02 02:54:05,349 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=afa57eddfa2d26bf35b12dc022351e78, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:05,349 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1682996044976.afa57eddfa2d26bf35b12dc022351e78.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1682996045349"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1682996045349"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1682996045349"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682996045349"}]},"ts":"1682996045349"} 2023-05-02 02:54:05,353 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=55, resume processing ppid=54 2023-05-02 02:54:05,353 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=55, ppid=54, state=SUCCESS; OpenRegionProcedure afa57eddfa2d26bf35b12dc022351e78, server=jenkins-hbase4.apache.org,42019,1682996021676 in 175 msec 2023-05-02 02:54:05,356 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=54, resume processing ppid=53 2023-05-02 02:54:05,356 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=54, ppid=53, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=afa57eddfa2d26bf35b12dc022351e78, ASSIGN in 332 msec 2023-05-02 02:54:05,357 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-05-02 02:54:05,357 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682996045357"}]},"ts":"1682996045357"} 2023-05-02 02:54:05,358 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLED in hbase:meta 2023-05-02 02:54:05,360 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_POST_OPERATION 2023-05-02 02:54:05,362 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile in 384 msec 2023-05-02 02:54:05,478 DEBUG [Listener at localhost/37119] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:05,479 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(1029): Stored pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=afa57eddfa2d26bf35b12dc022351e78, daughterA=b91a201af734ea9cb01587589c96e908, daughterB=292a312492e2395ec0ca83a160894be9 2023-05-02 02:54:05,480 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=afa57eddfa2d26bf35b12dc022351e78, daughterA=b91a201af734ea9cb01587589c96e908, daughterB=292a312492e2395ec0ca83a160894be9 2023-05-02 02:54:05,480 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=afa57eddfa2d26bf35b12dc022351e78, daughterA=b91a201af734ea9cb01587589c96e908, daughterB=292a312492e2395ec0ca83a160894be9 2023-05-02 02:54:05,480 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=afa57eddfa2d26bf35b12dc022351e78, daughterA=b91a201af734ea9cb01587589c96e908, daughterB=292a312492e2395ec0ca83a160894be9 2023-05-02 02:54:05,483 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=afa57eddfa2d26bf35b12dc022351e78, UNASSIGN}] 2023-05-02 02:54:05,483 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=afa57eddfa2d26bf35b12dc022351e78, UNASSIGN 2023-05-02 02:54:05,484 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=afa57eddfa2d26bf35b12dc022351e78, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:05,484 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1682996044976.afa57eddfa2d26bf35b12dc022351e78.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1682996045484"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1682996045484"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682996045484"}]},"ts":"1682996045484"} 2023-05-02 02:54:05,486 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=58, ppid=57, state=RUNNABLE; CloseRegionProcedure afa57eddfa2d26bf35b12dc022351e78, server=jenkins-hbase4.apache.org,42019,1682996021676}] 2023-05-02 02:54:05,638 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close afa57eddfa2d26bf35b12dc022351e78 2023-05-02 02:54:05,639 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing afa57eddfa2d26bf35b12dc022351e78, disabling compactions & flushes 2023-05-02 02:54:05,639 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionNoStoreFile,,1682996044976.afa57eddfa2d26bf35b12dc022351e78. 2023-05-02 02:54:05,639 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1682996044976.afa57eddfa2d26bf35b12dc022351e78. 2023-05-02 02:54:05,639 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionNoStoreFile,,1682996044976.afa57eddfa2d26bf35b12dc022351e78. after waiting 0 ms 2023-05-02 02:54:05,639 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionNoStoreFile,,1682996044976.afa57eddfa2d26bf35b12dc022351e78. 2023-05-02 02:54:05,643 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionNoStoreFile/afa57eddfa2d26bf35b12dc022351e78/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-05-02 02:54:05,643 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-02 02:54:05,643 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionNoStoreFile,,1682996044976.afa57eddfa2d26bf35b12dc022351e78. 2023-05-02 02:54:05,643 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for afa57eddfa2d26bf35b12dc022351e78: 2023-05-02 02:54:05,645 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed afa57eddfa2d26bf35b12dc022351e78 2023-05-02 02:54:05,645 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=afa57eddfa2d26bf35b12dc022351e78, regionState=CLOSED 2023-05-02 02:54:05,646 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1682996044976.afa57eddfa2d26bf35b12dc022351e78.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1682996045645"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682996045645"}]},"ts":"1682996045645"} 2023-05-02 02:54:05,648 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=58, resume processing ppid=57 2023-05-02 02:54:05,649 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=58, ppid=57, state=SUCCESS; CloseRegionProcedure afa57eddfa2d26bf35b12dc022351e78, server=jenkins-hbase4.apache.org,42019,1682996021676 in 161 msec 2023-05-02 02:54:05,650 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=57, resume processing ppid=56 2023-05-02 02:54:05,650 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=57, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=afa57eddfa2d26bf35b12dc022351e78, UNASSIGN in 166 msec 2023-05-02 02:54:05,679 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionNoStoreFile/b91a201af734ea9cb01587589c96e908/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-05-02 02:54:05,681 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionNoStoreFile/292a312492e2395ec0ca83a160894be9/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-05-02 02:54:05,682 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1682996044976.afa57eddfa2d26bf35b12dc022351e78.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1682996045682"},{"qualifier":"splitA","vlen":67,"tag":[],"timestamp":"1682996045682"},{"qualifier":"splitB","vlen":67,"tag":[],"timestamp":"1682996045682"}]},"ts":"1682996045682"} 2023-05-02 02:54:05,682 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1682996045478.b91a201af734ea9cb01587589c96e908.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1682996045682"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682996045682"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682996045682"}]},"ts":"1682996045682"} 2023-05-02 02:54:05,683 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1682996045478.292a312492e2395ec0ca83a160894be9.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1682996045682"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682996045682"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682996045682"}]},"ts":"1682996045682"} 2023-05-02 02:54:05,685 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=b91a201af734ea9cb01587589c96e908, ASSIGN}, {pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=292a312492e2395ec0ca83a160894be9, ASSIGN}] 2023-05-02 02:54:05,686 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=292a312492e2395ec0ca83a160894be9, ASSIGN 2023-05-02 02:54:05,687 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=292a312492e2395ec0ca83a160894be9, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,42019,1682996021676; forceNewPlan=false, retain=false 2023-05-02 02:54:05,687 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=b91a201af734ea9cb01587589c96e908, ASSIGN 2023-05-02 02:54:05,688 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=b91a201af734ea9cb01587589c96e908, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,42019,1682996021676; forceNewPlan=false, retain=false 2023-05-02 02:54:05,837 INFO [jenkins-hbase4:41119] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-05-02 02:54:05,839 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=b91a201af734ea9cb01587589c96e908, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:05,839 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1682996045478.b91a201af734ea9cb01587589c96e908.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1682996045839"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1682996045839"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682996045839"}]},"ts":"1682996045839"} 2023-05-02 02:54:05,841 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=61, ppid=59, state=RUNNABLE; OpenRegionProcedure b91a201af734ea9cb01587589c96e908, server=jenkins-hbase4.apache.org,42019,1682996021676}] 2023-05-02 02:54:05,843 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=292a312492e2395ec0ca83a160894be9, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:05,844 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1682996045478.292a312492e2395ec0ca83a160894be9.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1682996045843"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1682996045843"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682996045843"}]},"ts":"1682996045843"} 2023-05-02 02:54:05,845 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=62, ppid=60, state=RUNNABLE; OpenRegionProcedure 292a312492e2395ec0ca83a160894be9, server=jenkins-hbase4.apache.org,42019,1682996021676}] 2023-05-02 02:54:05,996 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionNoStoreFile,41,1682996045478.292a312492e2395ec0ca83a160894be9. 2023-05-02 02:54:05,996 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 292a312492e2395ec0ca83a160894be9, NAME => 'testSplitTableRegionNoStoreFile,41,1682996045478.292a312492e2395ec0ca83a160894be9.', STARTKEY => '41', ENDKEY => ''} 2023-05-02 02:54:05,996 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-02 02:54:05,996 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 292a312492e2395ec0ca83a160894be9 2023-05-02 02:54:05,996 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionNoStoreFile,41,1682996045478.292a312492e2395ec0ca83a160894be9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-02 02:54:05,996 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 292a312492e2395ec0ca83a160894be9 2023-05-02 02:54:05,996 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 292a312492e2395ec0ca83a160894be9 2023-05-02 02:54:05,997 INFO [StoreOpener-292a312492e2395ec0ca83a160894be9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 292a312492e2395ec0ca83a160894be9 2023-05-02 02:54:05,999 DEBUG [StoreOpener-292a312492e2395ec0ca83a160894be9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionNoStoreFile/292a312492e2395ec0ca83a160894be9/cf1 2023-05-02 02:54:05,999 DEBUG [StoreOpener-292a312492e2395ec0ca83a160894be9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionNoStoreFile/292a312492e2395ec0ca83a160894be9/cf1 2023-05-02 02:54:05,999 INFO [StoreOpener-292a312492e2395ec0ca83a160894be9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 292a312492e2395ec0ca83a160894be9 columnFamilyName cf1 2023-05-02 02:54:06,000 INFO [StoreOpener-292a312492e2395ec0ca83a160894be9-1] regionserver.HStore(310): Store=292a312492e2395ec0ca83a160894be9/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-02 02:54:06,000 INFO [StoreOpener-292a312492e2395ec0ca83a160894be9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 292a312492e2395ec0ca83a160894be9 2023-05-02 02:54:06,001 DEBUG [StoreOpener-292a312492e2395ec0ca83a160894be9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionNoStoreFile/292a312492e2395ec0ca83a160894be9/cf2 2023-05-02 02:54:06,001 DEBUG [StoreOpener-292a312492e2395ec0ca83a160894be9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionNoStoreFile/292a312492e2395ec0ca83a160894be9/cf2 2023-05-02 02:54:06,001 INFO [StoreOpener-292a312492e2395ec0ca83a160894be9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 292a312492e2395ec0ca83a160894be9 columnFamilyName cf2 2023-05-02 02:54:06,002 INFO [StoreOpener-292a312492e2395ec0ca83a160894be9-1] regionserver.HStore(310): Store=292a312492e2395ec0ca83a160894be9/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-02 02:54:06,002 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionNoStoreFile/292a312492e2395ec0ca83a160894be9 2023-05-02 02:54:06,003 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionNoStoreFile/292a312492e2395ec0ca83a160894be9 2023-05-02 02:54:06,005 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-02 02:54:06,006 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 292a312492e2395ec0ca83a160894be9 2023-05-02 02:54:06,006 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 292a312492e2395ec0ca83a160894be9; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68299280, jitterRate=0.01773858070373535}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-02 02:54:06,006 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 292a312492e2395ec0ca83a160894be9: 2023-05-02 02:54:06,007 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,41,1682996045478.292a312492e2395ec0ca83a160894be9., pid=62, masterSystemTime=1682996045993 2023-05-02 02:54:06,008 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,41,1682996045478.292a312492e2395ec0ca83a160894be9. 2023-05-02 02:54:06,008 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionNoStoreFile,41,1682996045478.292a312492e2395ec0ca83a160894be9. 2023-05-02 02:54:06,008 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionNoStoreFile,,1682996045478.b91a201af734ea9cb01587589c96e908. 2023-05-02 02:54:06,009 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => b91a201af734ea9cb01587589c96e908, NAME => 'testSplitTableRegionNoStoreFile,,1682996045478.b91a201af734ea9cb01587589c96e908.', STARTKEY => '', ENDKEY => '41'} 2023-05-02 02:54:06,009 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=292a312492e2395ec0ca83a160894be9, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:06,009 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-02 02:54:06,009 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile b91a201af734ea9cb01587589c96e908 2023-05-02 02:54:06,009 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionNoStoreFile,,1682996045478.b91a201af734ea9cb01587589c96e908.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-02 02:54:06,009 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,41,1682996045478.292a312492e2395ec0ca83a160894be9.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1682996046009"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1682996046009"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1682996046009"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682996046009"}]},"ts":"1682996046009"} 2023-05-02 02:54:06,009 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for b91a201af734ea9cb01587589c96e908 2023-05-02 02:54:06,009 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for b91a201af734ea9cb01587589c96e908 2023-05-02 02:54:06,010 INFO [StoreOpener-b91a201af734ea9cb01587589c96e908-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region b91a201af734ea9cb01587589c96e908 2023-05-02 02:54:06,012 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=62, resume processing ppid=60 2023-05-02 02:54:06,012 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=62, ppid=60, state=SUCCESS; OpenRegionProcedure 292a312492e2395ec0ca83a160894be9, server=jenkins-hbase4.apache.org,42019,1682996021676 in 165 msec 2023-05-02 02:54:06,012 DEBUG [StoreOpener-b91a201af734ea9cb01587589c96e908-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionNoStoreFile/b91a201af734ea9cb01587589c96e908/cf1 2023-05-02 02:54:06,012 DEBUG [StoreOpener-b91a201af734ea9cb01587589c96e908-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionNoStoreFile/b91a201af734ea9cb01587589c96e908/cf1 2023-05-02 02:54:06,012 INFO [StoreOpener-b91a201af734ea9cb01587589c96e908-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b91a201af734ea9cb01587589c96e908 columnFamilyName cf1 2023-05-02 02:54:06,013 INFO [StoreOpener-b91a201af734ea9cb01587589c96e908-1] regionserver.HStore(310): Store=b91a201af734ea9cb01587589c96e908/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-02 02:54:06,013 INFO [StoreOpener-b91a201af734ea9cb01587589c96e908-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region b91a201af734ea9cb01587589c96e908 2023-05-02 02:54:06,014 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=60, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=292a312492e2395ec0ca83a160894be9, ASSIGN in 327 msec 2023-05-02 02:54:06,016 DEBUG [StoreOpener-b91a201af734ea9cb01587589c96e908-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionNoStoreFile/b91a201af734ea9cb01587589c96e908/cf2 2023-05-02 02:54:06,016 DEBUG [StoreOpener-b91a201af734ea9cb01587589c96e908-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionNoStoreFile/b91a201af734ea9cb01587589c96e908/cf2 2023-05-02 02:54:06,016 INFO [StoreOpener-b91a201af734ea9cb01587589c96e908-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b91a201af734ea9cb01587589c96e908 columnFamilyName cf2 2023-05-02 02:54:06,017 INFO [StoreOpener-b91a201af734ea9cb01587589c96e908-1] regionserver.HStore(310): Store=b91a201af734ea9cb01587589c96e908/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-02 02:54:06,017 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionNoStoreFile/b91a201af734ea9cb01587589c96e908 2023-05-02 02:54:06,018 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionNoStoreFile/b91a201af734ea9cb01587589c96e908 2023-05-02 02:54:06,020 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-02 02:54:06,021 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for b91a201af734ea9cb01587589c96e908 2023-05-02 02:54:06,022 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened b91a201af734ea9cb01587589c96e908; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71561971, jitterRate=0.06635646522045135}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-02 02:54:06,022 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for b91a201af734ea9cb01587589c96e908: 2023-05-02 02:54:06,022 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1682996045478.b91a201af734ea9cb01587589c96e908., pid=61, masterSystemTime=1682996045993 2023-05-02 02:54:06,023 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1682996045478.b91a201af734ea9cb01587589c96e908. 2023-05-02 02:54:06,023 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionNoStoreFile,,1682996045478.b91a201af734ea9cb01587589c96e908. 2023-05-02 02:54:06,024 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=b91a201af734ea9cb01587589c96e908, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:06,024 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1682996045478.b91a201af734ea9cb01587589c96e908.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1682996046024"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1682996046024"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1682996046024"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682996046024"}]},"ts":"1682996046024"} 2023-05-02 02:54:06,027 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=61, resume processing ppid=59 2023-05-02 02:54:06,027 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=61, ppid=59, state=SUCCESS; OpenRegionProcedure b91a201af734ea9cb01587589c96e908, server=jenkins-hbase4.apache.org,42019,1682996021676 in 184 msec 2023-05-02 02:54:06,028 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=59, resume processing ppid=56 2023-05-02 02:54:06,028 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=59, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=b91a201af734ea9cb01587589c96e908, ASSIGN in 342 msec 2023-05-02 02:54:06,029 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=afa57eddfa2d26bf35b12dc022351e78, daughterA=b91a201af734ea9cb01587589c96e908, daughterB=292a312492e2395ec0ca83a160894be9 in 550 msec 2023-05-02 02:54:06,234 DEBUG [Listener at localhost/37119] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-05-02 02:54:06,237 WARN [Listener at localhost/37119] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-02 02:54:06,238 INFO [Listener at localhost/37119] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionNoStoreFile 2023-05-02 02:54:06,238 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testSplitTableRegionNoStoreFile 2023-05-02 02:54:06,239 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure2.ProcedureExecutor(1029): Stored pid=63, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-02 02:54:06,241 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-05-02 02:54:06,242 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682996046242"}]},"ts":"1682996046242"} 2023-05-02 02:54:06,243 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLING in hbase:meta 2023-05-02 02:54:06,248 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionNoStoreFile to state=DISABLING 2023-05-02 02:54:06,248 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=b91a201af734ea9cb01587589c96e908, UNASSIGN}, {pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=292a312492e2395ec0ca83a160894be9, UNASSIGN}] 2023-05-02 02:54:06,249 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=292a312492e2395ec0ca83a160894be9, UNASSIGN 2023-05-02 02:54:06,249 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=292a312492e2395ec0ca83a160894be9, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:06,250 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1682996045478.292a312492e2395ec0ca83a160894be9.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1682996046249"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1682996046249"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682996046249"}]},"ts":"1682996046249"} 2023-05-02 02:54:06,251 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=66, ppid=65, state=RUNNABLE; CloseRegionProcedure 292a312492e2395ec0ca83a160894be9, server=jenkins-hbase4.apache.org,42019,1682996021676}] 2023-05-02 02:54:06,253 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=b91a201af734ea9cb01587589c96e908, UNASSIGN 2023-05-02 02:54:06,253 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=b91a201af734ea9cb01587589c96e908, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:06,253 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1682996045478.b91a201af734ea9cb01587589c96e908.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1682996046253"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1682996046253"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682996046253"}]},"ts":"1682996046253"} 2023-05-02 02:54:06,254 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=67, ppid=64, state=RUNNABLE; CloseRegionProcedure b91a201af734ea9cb01587589c96e908, server=jenkins-hbase4.apache.org,42019,1682996021676}] 2023-05-02 02:54:06,343 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-05-02 02:54:06,403 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 292a312492e2395ec0ca83a160894be9 2023-05-02 02:54:06,404 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 292a312492e2395ec0ca83a160894be9, disabling compactions & flushes 2023-05-02 02:54:06,404 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionNoStoreFile,41,1682996045478.292a312492e2395ec0ca83a160894be9. 2023-05-02 02:54:06,404 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,41,1682996045478.292a312492e2395ec0ca83a160894be9. 2023-05-02 02:54:06,404 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionNoStoreFile,41,1682996045478.292a312492e2395ec0ca83a160894be9. after waiting 0 ms 2023-05-02 02:54:06,404 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionNoStoreFile,41,1682996045478.292a312492e2395ec0ca83a160894be9. 2023-05-02 02:54:06,408 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionNoStoreFile/292a312492e2395ec0ca83a160894be9/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-05-02 02:54:06,408 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-02 02:54:06,409 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionNoStoreFile,41,1682996045478.292a312492e2395ec0ca83a160894be9. 2023-05-02 02:54:06,409 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 292a312492e2395ec0ca83a160894be9: 2023-05-02 02:54:06,410 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 292a312492e2395ec0ca83a160894be9 2023-05-02 02:54:06,410 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close b91a201af734ea9cb01587589c96e908 2023-05-02 02:54:06,411 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing b91a201af734ea9cb01587589c96e908, disabling compactions & flushes 2023-05-02 02:54:06,411 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionNoStoreFile,,1682996045478.b91a201af734ea9cb01587589c96e908. 2023-05-02 02:54:06,411 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1682996045478.b91a201af734ea9cb01587589c96e908. 2023-05-02 02:54:06,411 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionNoStoreFile,,1682996045478.b91a201af734ea9cb01587589c96e908. after waiting 0 ms 2023-05-02 02:54:06,411 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionNoStoreFile,,1682996045478.b91a201af734ea9cb01587589c96e908. 2023-05-02 02:54:06,411 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=292a312492e2395ec0ca83a160894be9, regionState=CLOSED 2023-05-02 02:54:06,412 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,41,1682996045478.292a312492e2395ec0ca83a160894be9.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1682996046411"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682996046411"}]},"ts":"1682996046411"} 2023-05-02 02:54:06,415 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=66, resume processing ppid=65 2023-05-02 02:54:06,415 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=66, ppid=65, state=SUCCESS; CloseRegionProcedure 292a312492e2395ec0ca83a160894be9, server=jenkins-hbase4.apache.org,42019,1682996021676 in 162 msec 2023-05-02 02:54:06,415 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionNoStoreFile/b91a201af734ea9cb01587589c96e908/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-05-02 02:54:06,416 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-02 02:54:06,416 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionNoStoreFile,,1682996045478.b91a201af734ea9cb01587589c96e908. 2023-05-02 02:54:06,416 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for b91a201af734ea9cb01587589c96e908: 2023-05-02 02:54:06,416 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=65, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=292a312492e2395ec0ca83a160894be9, UNASSIGN in 167 msec 2023-05-02 02:54:06,417 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed b91a201af734ea9cb01587589c96e908 2023-05-02 02:54:06,417 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=b91a201af734ea9cb01587589c96e908, regionState=CLOSED 2023-05-02 02:54:06,417 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1682996045478.b91a201af734ea9cb01587589c96e908.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1682996046417"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682996046417"}]},"ts":"1682996046417"} 2023-05-02 02:54:06,420 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=67, resume processing ppid=64 2023-05-02 02:54:06,420 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=67, ppid=64, state=SUCCESS; CloseRegionProcedure b91a201af734ea9cb01587589c96e908, server=jenkins-hbase4.apache.org,42019,1682996021676 in 164 msec 2023-05-02 02:54:06,421 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=64, resume processing ppid=63 2023-05-02 02:54:06,422 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=64, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=b91a201af734ea9cb01587589c96e908, UNASSIGN in 172 msec 2023-05-02 02:54:06,422 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682996046422"}]},"ts":"1682996046422"} 2023-05-02 02:54:06,423 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLED in hbase:meta 2023-05-02 02:54:06,425 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionNoStoreFile to state=DISABLED 2023-05-02 02:54:06,426 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile in 186 msec 2023-05-02 02:54:06,544 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-05-02 02:54:06,544 INFO [Listener at localhost/37119] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 63 completed 2023-05-02 02:54:06,545 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testSplitTableRegionNoStoreFile 2023-05-02 02:54:06,546 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure2.ProcedureExecutor(1029): Stored pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-02 02:54:06,548 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-02 02:54:06,549 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=68, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-02 02:54:06,549 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1227): Checking to see if procedure is done pid=68 2023-05-02 02:54:06,553 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionNoStoreFile/afa57eddfa2d26bf35b12dc022351e78 2023-05-02 02:54:06,553 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionNoStoreFile/b91a201af734ea9cb01587589c96e908 2023-05-02 02:54:06,553 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionNoStoreFile/292a312492e2395ec0ca83a160894be9 2023-05-02 02:54:06,555 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionNoStoreFile/292a312492e2395ec0ca83a160894be9/cf1, FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionNoStoreFile/292a312492e2395ec0ca83a160894be9/cf2, FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionNoStoreFile/292a312492e2395ec0ca83a160894be9/recovered.edits] 2023-05-02 02:54:06,555 DEBUG [HFileArchiver-6] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionNoStoreFile/b91a201af734ea9cb01587589c96e908/cf1, FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionNoStoreFile/b91a201af734ea9cb01587589c96e908/cf2, FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionNoStoreFile/b91a201af734ea9cb01587589c96e908/recovered.edits] 2023-05-02 02:54:06,555 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionNoStoreFile/afa57eddfa2d26bf35b12dc022351e78/cf1, FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionNoStoreFile/afa57eddfa2d26bf35b12dc022351e78/cf2, FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionNoStoreFile/afa57eddfa2d26bf35b12dc022351e78/recovered.edits] 2023-05-02 02:54:06,562 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionNoStoreFile/292a312492e2395ec0ca83a160894be9/recovered.edits/7.seqid to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionNoStoreFile/292a312492e2395ec0ca83a160894be9/recovered.edits/7.seqid 2023-05-02 02:54:06,562 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionNoStoreFile/afa57eddfa2d26bf35b12dc022351e78/recovered.edits/4.seqid to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionNoStoreFile/afa57eddfa2d26bf35b12dc022351e78/recovered.edits/4.seqid 2023-05-02 02:54:06,562 DEBUG [HFileArchiver-6] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionNoStoreFile/b91a201af734ea9cb01587589c96e908/recovered.edits/7.seqid to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionNoStoreFile/b91a201af734ea9cb01587589c96e908/recovered.edits/7.seqid 2023-05-02 02:54:06,563 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionNoStoreFile/292a312492e2395ec0ca83a160894be9 2023-05-02 02:54:06,563 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionNoStoreFile/afa57eddfa2d26bf35b12dc022351e78 2023-05-02 02:54:06,563 DEBUG [HFileArchiver-6] backup.HFileArchiver(596): Deleted hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionNoStoreFile/b91a201af734ea9cb01587589c96e908 2023-05-02 02:54:06,563 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-05-02 02:54:06,565 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=68, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-02 02:54:06,567 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionNoStoreFile from hbase:meta 2023-05-02 02:54:06,569 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionNoStoreFile' descriptor. 2023-05-02 02:54:06,570 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=68, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-02 02:54:06,570 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionNoStoreFile' from region states. 2023-05-02 02:54:06,571 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1682996044976.afa57eddfa2d26bf35b12dc022351e78.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1682996046570"}]},"ts":"9223372036854775807"} 2023-05-02 02:54:06,571 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1682996045478.b91a201af734ea9cb01587589c96e908.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1682996046570"}]},"ts":"9223372036854775807"} 2023-05-02 02:54:06,571 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,41,1682996045478.292a312492e2395ec0ca83a160894be9.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1682996046570"}]},"ts":"9223372036854775807"} 2023-05-02 02:54:06,572 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-05-02 02:54:06,572 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => afa57eddfa2d26bf35b12dc022351e78, NAME => 'testSplitTableRegionNoStoreFile,,1682996044976.afa57eddfa2d26bf35b12dc022351e78.', STARTKEY => '', ENDKEY => ''}, {ENCODED => b91a201af734ea9cb01587589c96e908, NAME => 'testSplitTableRegionNoStoreFile,,1682996045478.b91a201af734ea9cb01587589c96e908.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 292a312492e2395ec0ca83a160894be9, NAME => 'testSplitTableRegionNoStoreFile,41,1682996045478.292a312492e2395ec0ca83a160894be9.', STARTKEY => '41', ENDKEY => ''}] 2023-05-02 02:54:06,572 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionNoStoreFile' as deleted. 2023-05-02 02:54:06,573 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1682996046572"}]},"ts":"9223372036854775807"} 2023-05-02 02:54:06,574 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionNoStoreFile state from META 2023-05-02 02:54:06,576 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=68, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-02 02:54:06,577 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile in 31 msec 2023-05-02 02:54:06,650 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1227): Checking to see if procedure is done pid=68 2023-05-02 02:54:06,650 INFO [Listener at localhost/37119] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 68 completed 2023-05-02 02:54:06,665 INFO [Listener at localhost/37119] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=406 (was 402) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1888261172_15 at /127.0.0.1:40234 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2c67b29-shared-pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2c67b29-metaLookup-shared--pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2c67b29-shared-pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2c67b29-metaLookup-shared--pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1888261172_15 at /127.0.0.1:59682 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=867 (was 867), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=268 (was 268), ProcessCount=170 (was 170), AvailableMemoryMB=13853 (was 13853) 2023-05-02 02:54:06,679 INFO [Listener at localhost/37119] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=406, OpenFileDescriptor=867, MaxFileDescriptor=60000, SystemLoadAverage=268, ProcessCount=170, AvailableMemoryMB=13852 2023-05-02 02:54:06,679 WARN [Listener at localhost/37119] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-02 02:54:06,681 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-05-02 02:54:06,681 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-05-02 02:54:06,682 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(1029): Stored pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:06,684 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-05-02 02:54:06,685 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-05-02 02:54:06,686 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:06,687 DEBUG [HFileArchiver-8] backup.HFileArchiver(153): Directory hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f empty. 2023-05-02 02:54:06,687 DEBUG [HFileArchiver-8] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:06,687 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-05-02 02:54:06,697 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/.tabledesc/.tableinfo.0000000001 2023-05-02 02:54:06,698 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(7675): creating {ENCODED => 8af7d4111b21c70efea08f7beab73a9f, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1682996046681.8af7d4111b21c70efea08f7beab73a9f.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionDeletedRowsDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp 2023-05-02 02:54:06,706 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegionDeletedRowsDaughter,,1682996046681.8af7d4111b21c70efea08f7beab73a9f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-02 02:54:06,707 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1604): Closing 8af7d4111b21c70efea08f7beab73a9f, disabling compactions & flushes 2023-05-02 02:54:06,707 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegionDeletedRowsDaughter,,1682996046681.8af7d4111b21c70efea08f7beab73a9f. 2023-05-02 02:54:06,707 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1682996046681.8af7d4111b21c70efea08f7beab73a9f. 2023-05-02 02:54:06,707 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1682996046681.8af7d4111b21c70efea08f7beab73a9f. after waiting 0 ms 2023-05-02 02:54:06,707 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1682996046681.8af7d4111b21c70efea08f7beab73a9f. 2023-05-02 02:54:06,707 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegionDeletedRowsDaughter,,1682996046681.8af7d4111b21c70efea08f7beab73a9f. 2023-05-02 02:54:06,707 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1558): Region close journal for 8af7d4111b21c70efea08f7beab73a9f: 2023-05-02 02:54:06,709 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-05-02 02:54:06,710 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1682996046681.8af7d4111b21c70efea08f7beab73a9f.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1682996046710"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682996046710"}]},"ts":"1682996046710"} 2023-05-02 02:54:06,711 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-05-02 02:54:06,712 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-05-02 02:54:06,712 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682996046712"}]},"ts":"1682996046712"} 2023-05-02 02:54:06,713 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLING in hbase:meta 2023-05-02 02:54:06,716 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-05-02 02:54:06,717 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-05-02 02:54:06,717 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-05-02 02:54:06,717 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-05-02 02:54:06,717 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-05-02 02:54:06,717 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=8af7d4111b21c70efea08f7beab73a9f, ASSIGN}] 2023-05-02 02:54:06,719 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=8af7d4111b21c70efea08f7beab73a9f, ASSIGN 2023-05-02 02:54:06,719 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=8af7d4111b21c70efea08f7beab73a9f, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,39081,1682996021579; forceNewPlan=false, retain=false 2023-05-02 02:54:06,869 INFO [jenkins-hbase4:41119] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-05-02 02:54:06,871 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=8af7d4111b21c70efea08f7beab73a9f, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:06,871 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1682996046681.8af7d4111b21c70efea08f7beab73a9f.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1682996046871"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1682996046871"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682996046871"}]},"ts":"1682996046871"} 2023-05-02 02:54:06,873 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=71, ppid=70, state=RUNNABLE; OpenRegionProcedure 8af7d4111b21c70efea08f7beab73a9f, server=jenkins-hbase4.apache.org,39081,1682996021579}] 2023-05-02 02:54:07,027 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionDeletedRowsDaughter,,1682996046681.8af7d4111b21c70efea08f7beab73a9f. 2023-05-02 02:54:07,027 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 8af7d4111b21c70efea08f7beab73a9f, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1682996046681.8af7d4111b21c70efea08f7beab73a9f.', STARTKEY => '', ENDKEY => ''} 2023-05-02 02:54:07,028 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-02 02:54:07,028 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter 8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:07,028 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionDeletedRowsDaughter,,1682996046681.8af7d4111b21c70efea08f7beab73a9f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-02 02:54:07,028 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:07,028 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:07,029 INFO [StoreOpener-8af7d4111b21c70efea08f7beab73a9f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:07,031 DEBUG [StoreOpener-8af7d4111b21c70efea08f7beab73a9f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1 2023-05-02 02:54:07,031 DEBUG [StoreOpener-8af7d4111b21c70efea08f7beab73a9f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1 2023-05-02 02:54:07,031 INFO [StoreOpener-8af7d4111b21c70efea08f7beab73a9f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8af7d4111b21c70efea08f7beab73a9f columnFamilyName cf1 2023-05-02 02:54:07,032 INFO [StoreOpener-8af7d4111b21c70efea08f7beab73a9f-1] regionserver.HStore(310): Store=8af7d4111b21c70efea08f7beab73a9f/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-02 02:54:07,032 INFO [StoreOpener-8af7d4111b21c70efea08f7beab73a9f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:07,033 DEBUG [StoreOpener-8af7d4111b21c70efea08f7beab73a9f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2 2023-05-02 02:54:07,033 DEBUG [StoreOpener-8af7d4111b21c70efea08f7beab73a9f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2 2023-05-02 02:54:07,034 INFO [StoreOpener-8af7d4111b21c70efea08f7beab73a9f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8af7d4111b21c70efea08f7beab73a9f columnFamilyName cf2 2023-05-02 02:54:07,035 INFO [StoreOpener-8af7d4111b21c70efea08f7beab73a9f-1] regionserver.HStore(310): Store=8af7d4111b21c70efea08f7beab73a9f/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-02 02:54:07,036 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:07,036 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:07,039 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-02 02:54:07,041 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:07,043 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-05-02 02:54:07,044 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 8af7d4111b21c70efea08f7beab73a9f; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63737087, jitterRate=-0.05024339258670807}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-02 02:54:07,044 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 8af7d4111b21c70efea08f7beab73a9f: 2023-05-02 02:54:07,045 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1682996046681.8af7d4111b21c70efea08f7beab73a9f., pid=71, masterSystemTime=1682996047024 2023-05-02 02:54:07,046 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1682996046681.8af7d4111b21c70efea08f7beab73a9f. 2023-05-02 02:54:07,046 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionDeletedRowsDaughter,,1682996046681.8af7d4111b21c70efea08f7beab73a9f. 2023-05-02 02:54:07,047 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=8af7d4111b21c70efea08f7beab73a9f, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,047 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1682996046681.8af7d4111b21c70efea08f7beab73a9f.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1682996047047"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1682996047047"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1682996047047"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682996047047"}]},"ts":"1682996047047"} 2023-05-02 02:54:07,050 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=71, resume processing ppid=70 2023-05-02 02:54:07,050 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=71, ppid=70, state=SUCCESS; OpenRegionProcedure 8af7d4111b21c70efea08f7beab73a9f, server=jenkins-hbase4.apache.org,39081,1682996021579 in 175 msec 2023-05-02 02:54:07,053 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=70, resume processing ppid=69 2023-05-02 02:54:07,053 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=70, ppid=69, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=8af7d4111b21c70efea08f7beab73a9f, ASSIGN in 333 msec 2023-05-02 02:54:07,054 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-05-02 02:54:07,054 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682996047054"}]},"ts":"1682996047054"} 2023-05-02 02:54:07,055 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLED in hbase:meta 2023-05-02 02:54:07,057 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-05-02 02:54:07,059 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 376 msec 2023-05-02 02:54:07,190 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-02 02:54:07,192 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,192 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-05-02 02:54:07,192 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:54:07,192 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-05-02 02:54:07,192 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-05-02 02:54:07,193 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,193 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,194 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:54:07,194 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:54:07,194 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:54:07,194 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:54:07,194 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:54:07,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:54:07,194 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:54:07,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:54:07,194 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:54:07,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,195 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,195 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-02 02:54:07,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,195 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,195 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,195 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:54:07,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,196 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:54:07,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,196 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:54:07,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,196 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,196 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,196 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-02 02:54:07,196 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-02 02:54:07,196 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:54:07,196 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,196 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-02 02:54:07,196 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-02 02:54:07,196 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38925,1682996021633' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-02 02:54:07,196 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:54:07,196 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-02 02:54:07,196 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-02 02:54:07,196 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42019,1682996021676' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-02 02:54:07,201 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,201 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-02 02:54:07,201 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:54:07,201 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-02 02:54:07,203 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-02 02:54:07,203 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-02 02:54:07,203 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1682996046681.8af7d4111b21c70efea08f7beab73a9f. 2023-05-02 02:54:07,203 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,203 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1682996046681.8af7d4111b21c70efea08f7beab73a9f. started... 2023-05-02 02:54:07,203 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,204 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8af7d4111b21c70efea08f7beab73a9f 2/2 column families, dataSize=120 B heapSize=928 B 2023-05-02 02:54:07,204 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,204 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:54:07,204 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,204 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:54:07,226 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/.tmp/cf1/3189d0758d394d4290511c7d95e297bb 2023-05-02 02:54:07,246 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/.tmp/cf2/76b4e49972d544bdb7a8702299d57cc6 2023-05-02 02:54:07,251 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/.tmp/cf1/3189d0758d394d4290511c7d95e297bb as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/3189d0758d394d4290511c7d95e297bb 2023-05-02 02:54:07,256 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/3189d0758d394d4290511c7d95e297bb, entries=2, sequenceid=6, filesize=4.8 K 2023-05-02 02:54:07,257 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/.tmp/cf2/76b4e49972d544bdb7a8702299d57cc6 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/76b4e49972d544bdb7a8702299d57cc6 2023-05-02 02:54:07,261 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/76b4e49972d544bdb7a8702299d57cc6, entries=2, sequenceid=6, filesize=4.8 K 2023-05-02 02:54:07,262 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 8af7d4111b21c70efea08f7beab73a9f in 58ms, sequenceid=6, compaction requested=false 2023-05-02 02:54:07,262 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionDeletedRowsDaughter' 2023-05-02 02:54:07,263 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8af7d4111b21c70efea08f7beab73a9f: 2023-05-02 02:54:07,263 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1682996046681.8af7d4111b21c70efea08f7beab73a9f. 2023-05-02 02:54:07,263 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-02 02:54:07,263 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-02 02:54:07,263 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,263 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-02 02:54:07,263 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-02 02:54:07,264 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,264 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:54:07,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:54:07,265 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,265 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:54:07,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:54:07,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:54:07,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:07,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:07,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:54:07,267 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-05-02 02:54:07,267 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@70463b13[Count = 0] remaining members to acquire global barrier 2023-05-02 02:54:07,267 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-05-02 02:54:07,267 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,268 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,268 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,268 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,268 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,268 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-02 02:54:07,268 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-02 02:54:07,268 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,269 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-02 02:54:07,268 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,268 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,38925,1682996021633' in zk 2023-05-02 02:54:07,268 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,269 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-02 02:54:07,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,269 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-02 02:54:07,269 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-02 02:54:07,269 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-02 02:54:07,269 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,39081,1682996021579' in zk 2023-05-02 02:54:07,269 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,42019,1682996021676' in zk 2023-05-02 02:54:07,270 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-02 02:54:07,270 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:54:07,270 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-02 02:54:07,272 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-02 02:54:07,272 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:54:07,272 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-02 02:54:07,273 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-02 02:54:07,273 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,273 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:54:07,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:54:07,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:54:07,273 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-02 02:54:07,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:54:07,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:54:07,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:07,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:07,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:54:07,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:07,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:07,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,39081,1682996021579': 2023-05-02 02:54:07,276 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-05-02 02:54:07,276 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-05-02 02:54:07,276 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-02 02:54:07,276 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-02 02:54:07,276 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,276 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-02 02:54:07,278 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,278 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:54:07,278 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,278 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,278 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,278 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:54:07,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,278 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:54:07,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:54:07,278 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,279 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:54:07,279 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,279 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:54:07,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:54:07,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:54:07,279 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:07,279 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:54:07,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:54:07,279 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:54:07,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:54:07,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,280 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:07,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:54:07,280 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:07,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:07,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:54:07,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,283 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:07,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:07,283 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:07,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:07,283 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,297 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,297 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:54:07,297 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:54:07,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:54:07,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:54:07,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:54:07,297 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:54:07,297 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:54:07,297 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,297 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:54:07,297 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:54:07,298 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:07,298 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:54:07,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:54:07,297 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-05-02 02:54:07,298 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-02 02:54:07,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:54:07,298 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:07,298 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:54:07,298 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:54:07,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:54:07,298 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-05-02 02:54:07,298 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,298 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-02 02:54:07,299 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:54:07,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:54:07,298 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,298 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:54:07,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:54:07,299 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,299 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:07,299 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:07,299 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,299 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,399 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-02 02:54:07,400 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-02 02:54:07,414 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-02 02:54:07,416 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-05-02 02:54:07,416 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,416 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-05-02 02:54:07,416 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:54:07,416 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-05-02 02:54:07,416 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-05-02 02:54:07,416 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,416 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,418 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:54:07,418 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:54:07,418 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:54:07,418 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:54:07,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:54:07,418 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:54:07,418 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,418 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:54:07,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:54:07,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:54:07,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,418 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,418 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-02 02:54:07,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,419 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,419 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,419 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:54:07,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,419 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:54:07,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,419 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-02 02:54:07,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:54:07,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,419 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,419 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,419 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,420 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-02 02:54:07,420 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:54:07,420 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-02 02:54:07,420 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-02 02:54:07,420 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42019,1682996021676' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-02 02:54:07,420 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-02 02:54:07,420 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-02 02:54:07,420 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,420 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,420 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-02 02:54:07,420 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:54:07,420 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-02 02:54:07,420 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:54:07,420 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-02 02:54:07,420 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-02 02:54:07,420 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-02 02:54:07,420 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38925,1682996021633' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-02 02:54:07,420 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-02 02:54:07,420 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-02 02:54:07,421 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1682996046681.8af7d4111b21c70efea08f7beab73a9f. 2023-05-02 02:54:07,421 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1682996046681.8af7d4111b21c70efea08f7beab73a9f. started... 2023-05-02 02:54:07,421 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8af7d4111b21c70efea08f7beab73a9f 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-02 02:54:07,421 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,421 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,422 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:54:07,422 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,423 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,423 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:54:07,431 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/.tmp/cf1/f6fbf066cf0a4af6b2ea76697cc904da 2023-05-02 02:54:07,443 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/.tmp/cf2/e6bbd81d56884f75a2ab34f638d3dc7f 2023-05-02 02:54:07,447 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/.tmp/cf1/f6fbf066cf0a4af6b2ea76697cc904da as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/f6fbf066cf0a4af6b2ea76697cc904da 2023-05-02 02:54:07,451 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/f6fbf066cf0a4af6b2ea76697cc904da, entries=10, sequenceid=19, filesize=5.0 K 2023-05-02 02:54:07,451 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/.tmp/cf2/e6bbd81d56884f75a2ab34f638d3dc7f as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/e6bbd81d56884f75a2ab34f638d3dc7f 2023-05-02 02:54:07,455 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/e6bbd81d56884f75a2ab34f638d3dc7f, entries=10, sequenceid=19, filesize=5.0 K 2023-05-02 02:54:07,456 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 8af7d4111b21c70efea08f7beab73a9f in 35ms, sequenceid=19, compaction requested=false 2023-05-02 02:54:07,456 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8af7d4111b21c70efea08f7beab73a9f: 2023-05-02 02:54:07,456 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1682996046681.8af7d4111b21c70efea08f7beab73a9f. 2023-05-02 02:54:07,456 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-02 02:54:07,456 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-02 02:54:07,456 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,456 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-02 02:54:07,456 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-02 02:54:07,459 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,459 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:54:07,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:54:07,459 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,459 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:54:07,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:54:07,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:54:07,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:07,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:07,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:54:07,461 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-05-02 02:54:07,461 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@68680ee5[Count = 0] remaining members to acquire global barrier 2023-05-02 02:54:07,461 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-05-02 02:54:07,461 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,462 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,462 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,462 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,462 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,462 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-02 02:54:07,462 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-02 02:54:07,462 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,462 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-02 02:54:07,462 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,462 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,462 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-02 02:54:07,462 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-02 02:54:07,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,462 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,42019,1682996021676' in zk 2023-05-02 02:54:07,462 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-02 02:54:07,462 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-02 02:54:07,462 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,39081,1682996021579' in zk 2023-05-02 02:54:07,462 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,38925,1682996021633' in zk 2023-05-02 02:54:07,464 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:54:07,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:54:07,464 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-02 02:54:07,464 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-02 02:54:07,464 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-02 02:54:07,464 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:54:07,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:54:07,464 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:54:07,464 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-02 02:54:07,464 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-02 02:54:07,464 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:54:07,464 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-02 02:54:07,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:54:07,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:07,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:07,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:54:07,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:07,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:07,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,39081,1682996021579': 2023-05-02 02:54:07,467 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-05-02 02:54:07,467 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-05-02 02:54:07,467 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-02 02:54:07,467 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-02 02:54:07,467 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,467 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-02 02:54:07,468 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,468 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:54:07,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:54:07,468 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,468 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,468 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:54:07,468 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:54:07,468 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,468 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,468 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,468 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:54:07,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,469 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:07,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:54:07,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,469 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:07,469 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:54:07,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:54:07,469 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:54:07,469 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:54:07,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,469 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:54:07,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:54:07,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:54:07,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:07,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:07,471 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:07,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,471 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:07,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:54:07,471 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:07,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:07,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,475 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,475 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:54:07,475 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,475 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:54:07,475 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:07,475 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-05-02 02:54:07,475 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:54:07,475 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:54:07,475 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:54:07,475 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:54:07,475 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:54:07,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:54:07,475 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:54:07,475 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-02 02:54:07,475 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:07,475 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:54:07,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:54:07,475 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:54:07,476 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:54:07,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:54:07,476 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:54:07,476 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,476 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-05-02 02:54:07,476 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,476 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-02 02:54:07,476 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:07,476 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:54:07,476 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:07,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:54:07,476 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,476 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:54:07,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:54:07,476 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,576 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-02 02:54:07,577 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-02 02:54:07,590 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-02 02:54:07,591 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-05-02 02:54:07,591 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,591 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-05-02 02:54:07,591 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:54:07,591 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-05-02 02:54:07,591 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-05-02 02:54:07,592 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,592 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,593 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:54:07,593 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:54:07,593 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:54:07,593 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:54:07,593 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,593 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:54:07,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:54:07,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:54:07,593 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:54:07,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:54:07,593 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,594 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-02 02:54:07,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,594 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,594 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,594 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:54:07,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:54:07,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,594 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,594 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:54:07,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,594 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,595 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-02 02:54:07,595 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,595 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-02 02:54:07,595 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-02 02:54:07,595 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,595 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-02 02:54:07,595 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-02 02:54:07,595 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,595 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:54:07,595 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-02 02:54:07,595 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-02 02:54:07,595 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-02 02:54:07,595 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42019,1682996021676' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-02 02:54:07,595 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:54:07,595 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:54:07,595 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-02 02:54:07,595 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-02 02:54:07,595 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-02 02:54:07,595 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38925,1682996021633' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-02 02:54:07,595 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-02 02:54:07,595 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-02 02:54:07,595 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1682996046681.8af7d4111b21c70efea08f7beab73a9f. 2023-05-02 02:54:07,595 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1682996046681.8af7d4111b21c70efea08f7beab73a9f. started... 2023-05-02 02:54:07,595 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8af7d4111b21c70efea08f7beab73a9f 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-02 02:54:07,596 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,597 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,597 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,597 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:54:07,597 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,597 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:54:07,608 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/.tmp/cf1/fa00f661973547868b8b686731148b94 2023-05-02 02:54:07,621 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/.tmp/cf2/eecb4f6bc8094e669b2ee7981bc5a39f 2023-05-02 02:54:07,625 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/.tmp/cf1/fa00f661973547868b8b686731148b94 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/fa00f661973547868b8b686731148b94 2023-05-02 02:54:07,629 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/fa00f661973547868b8b686731148b94, entries=10, sequenceid=32, filesize=5.0 K 2023-05-02 02:54:07,629 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/.tmp/cf2/eecb4f6bc8094e669b2ee7981bc5a39f as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/eecb4f6bc8094e669b2ee7981bc5a39f 2023-05-02 02:54:07,633 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/eecb4f6bc8094e669b2ee7981bc5a39f, entries=10, sequenceid=32, filesize=5.0 K 2023-05-02 02:54:07,634 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 8af7d4111b21c70efea08f7beab73a9f in 39ms, sequenceid=32, compaction requested=true 2023-05-02 02:54:07,634 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8af7d4111b21c70efea08f7beab73a9f: 2023-05-02 02:54:07,634 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1682996046681.8af7d4111b21c70efea08f7beab73a9f. 2023-05-02 02:54:07,634 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-02 02:54:07,634 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-02 02:54:07,634 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,634 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-02 02:54:07,634 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-02 02:54:07,639 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,639 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:54:07,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:54:07,639 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,639 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:54:07,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:54:07,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:54:07,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:07,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:07,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:54:07,641 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-05-02 02:54:07,641 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3c8da136[Count = 0] remaining members to acquire global barrier 2023-05-02 02:54:07,641 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-05-02 02:54:07,641 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,642 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,642 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,642 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,642 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,642 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-02 02:54:07,642 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-02 02:54:07,642 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,642 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-02 02:54:07,642 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,642 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,38925,1682996021633' in zk 2023-05-02 02:54:07,642 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,642 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-02 02:54:07,643 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-02 02:54:07,643 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-02 02:54:07,643 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-02 02:54:07,643 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,39081,1682996021579' in zk 2023-05-02 02:54:07,643 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,42019,1682996021676' in zk 2023-05-02 02:54:07,644 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-02 02:54:07,644 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:54:07,644 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-02 02:54:07,645 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-02 02:54:07,645 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-02 02:54:07,645 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,645 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:54:07,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:54:07,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:54:07,645 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-02 02:54:07,645 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:54:07,645 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-02 02:54:07,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:54:07,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:54:07,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:07,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:07,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:54:07,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:07,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:07,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,39081,1682996021579': 2023-05-02 02:54:07,648 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-05-02 02:54:07,648 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-05-02 02:54:07,648 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-02 02:54:07,648 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-02 02:54:07,648 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,648 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-02 02:54:07,650 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,650 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,650 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,650 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:54:07,650 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,650 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,650 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:54:07,650 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,650 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,650 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:54:07,650 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:07,650 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:54:07,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:54:07,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:54:07,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:54:07,650 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:54:07,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:54:07,651 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:54:07,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:54:07,651 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:07,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:54:07,651 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:54:07,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:07,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:07,653 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:07,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,653 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:07,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:54:07,653 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:07,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:07,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,656 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,656 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:54:07,656 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:54:07,656 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:54:07,656 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:54:07,656 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:54:07,656 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:54:07,656 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:54:07,656 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:54:07,656 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,656 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:54:07,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:54:07,656 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:54:07,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:54:07,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:54:07,656 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-05-02 02:54:07,656 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-02 02:54:07,656 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:07,656 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:54:07,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:54:07,656 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:07,656 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:54:07,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:54:07,657 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,657 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-05-02 02:54:07,656 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:54:07,657 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-02 02:54:07,657 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:54:07,657 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,657 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:07,657 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:07,657 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,657 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,757 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-02 02:54:07,758 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-02 02:54:07,771 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-02 02:54:07,772 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-05-02 02:54:07,772 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,772 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-05-02 02:54:07,772 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:54:07,772 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-05-02 02:54:07,772 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-05-02 02:54:07,773 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,773 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,774 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,774 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:54:07,774 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:54:07,774 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:54:07,774 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:54:07,774 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:54:07,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:54:07,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:54:07,774 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:54:07,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:54:07,774 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,775 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-02 02:54:07,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,775 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,775 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,775 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:54:07,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:54:07,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:54:07,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,776 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,776 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,776 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,776 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-02 02:54:07,776 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,776 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-02 02:54:07,776 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:54:07,776 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-02 02:54:07,776 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-02 02:54:07,776 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42019,1682996021676' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-02 02:54:07,777 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-02 02:54:07,777 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,777 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-02 02:54:07,777 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:54:07,777 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-02 02:54:07,777 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-02 02:54:07,777 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38925,1682996021633' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-02 02:54:07,777 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-02 02:54:07,777 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,777 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-02 02:54:07,777 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:54:07,777 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-02 02:54:07,777 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-02 02:54:07,777 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-02 02:54:07,777 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1682996046681.8af7d4111b21c70efea08f7beab73a9f. 2023-05-02 02:54:07,777 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1682996046681.8af7d4111b21c70efea08f7beab73a9f. started... 2023-05-02 02:54:07,778 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8af7d4111b21c70efea08f7beab73a9f 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-02 02:54:07,778 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,780 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,780 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,781 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:54:07,781 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,781 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:54:07,789 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/.tmp/cf1/b0c982e0384d423aa455befed4ca21c3 2023-05-02 02:54:07,801 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/.tmp/cf2/f894abdeb02b4e87bc8789e67b483cc0 2023-05-02 02:54:07,805 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/.tmp/cf1/b0c982e0384d423aa455befed4ca21c3 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/b0c982e0384d423aa455befed4ca21c3 2023-05-02 02:54:07,809 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/b0c982e0384d423aa455befed4ca21c3, entries=10, sequenceid=45, filesize=5.0 K 2023-05-02 02:54:07,810 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/.tmp/cf2/f894abdeb02b4e87bc8789e67b483cc0 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/f894abdeb02b4e87bc8789e67b483cc0 2023-05-02 02:54:07,815 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/f894abdeb02b4e87bc8789e67b483cc0, entries=10, sequenceid=45, filesize=5.0 K 2023-05-02 02:54:07,817 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 8af7d4111b21c70efea08f7beab73a9f in 39ms, sequenceid=45, compaction requested=true 2023-05-02 02:54:07,817 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8af7d4111b21c70efea08f7beab73a9f: 2023-05-02 02:54:07,817 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1682996046681.8af7d4111b21c70efea08f7beab73a9f. 2023-05-02 02:54:07,817 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-02 02:54:07,817 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-02 02:54:07,817 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,817 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-02 02:54:07,817 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-02 02:54:07,818 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,819 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:54:07,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:54:07,819 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,819 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:54:07,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:54:07,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:54:07,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:07,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:07,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:54:07,821 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-05-02 02:54:07,821 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@27c07113[Count = 0] remaining members to acquire global barrier 2023-05-02 02:54:07,821 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-05-02 02:54:07,821 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,822 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,823 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,822 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,823 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,823 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,823 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,823 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-02 02:54:07,823 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-02 02:54:07,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,823 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,823 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-02 02:54:07,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,823 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-02 02:54:07,823 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-02 02:54:07,823 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,42019,1682996021676' in zk 2023-05-02 02:54:07,823 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-02 02:54:07,823 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,38925,1682996021633' in zk 2023-05-02 02:54:07,823 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-02 02:54:07,823 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,39081,1682996021579' in zk 2023-05-02 02:54:07,825 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-02 02:54:07,825 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:54:07,825 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-02 02:54:07,827 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-02 02:54:07,827 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,827 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:54:07,827 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-02 02:54:07,827 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-02 02:54:07,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:54:07,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:54:07,827 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:54:07,827 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-02 02:54:07,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:54:07,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:54:07,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:07,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:07,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:54:07,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:07,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:07,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,39081,1682996021579': 2023-05-02 02:54:07,830 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-05-02 02:54:07,830 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-05-02 02:54:07,831 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-02 02:54:07,831 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-02 02:54:07,831 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,831 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-02 02:54:07,835 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,835 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,835 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,835 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:54:07,835 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:54:07,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,835 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,835 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:54:07,835 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:54:07,836 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,836 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:54:07,836 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:54:07,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:54:07,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,836 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:54:07,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:54:07,836 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:07,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,836 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:54:07,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:54:07,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:54:07,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,837 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:07,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,837 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:54:07,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:07,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:07,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,839 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:07,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:54:07,839 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:07,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,839 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:07,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:07,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,842 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:54:07,842 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,842 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:54:07,842 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,842 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:54:07,842 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:54:07,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:54:07,842 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:54:07,842 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-05-02 02:54:07,843 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-02 02:54:07,842 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:54:07,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:54:07,843 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:54:07,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:54:07,842 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:07,842 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:54:07,842 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:54:07,843 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:54:07,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:54:07,843 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-05-02 02:54:07,843 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:07,843 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:54:07,843 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,843 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:54:07,843 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-02 02:54:07,843 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:54:07,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:54:07,843 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:54:07,843 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,844 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:07,844 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:07,844 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,844 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,943 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-02 02:54:07,945 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-02 02:54:07,956 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-02 02:54:07,958 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-05-02 02:54:07,958 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,958 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-05-02 02:54:07,958 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:54:07,958 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-05-02 02:54:07,958 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-05-02 02:54:07,959 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,959 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,961 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:54:07,961 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,961 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:54:07,961 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:54:07,961 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:54:07,961 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:54:07,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:54:07,961 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:07,961 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-02 02:54:07,961 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:54:07,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:54:07,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:54:07,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,962 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,962 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,962 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:54:07,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,962 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:54:07,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:54:07,963 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-02 02:54:07,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,963 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,963 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,963 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,963 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-02 02:54:07,963 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:54:07,963 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-02 02:54:07,963 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-02 02:54:07,963 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-02 02:54:07,963 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-02 02:54:07,963 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-02 02:54:07,963 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1682996046681.8af7d4111b21c70efea08f7beab73a9f. 2023-05-02 02:54:07,963 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,963 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1682996046681.8af7d4111b21c70efea08f7beab73a9f. started... 2023-05-02 02:54:07,963 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-02 02:54:07,963 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:54:07,963 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,963 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-02 02:54:07,963 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8af7d4111b21c70efea08f7beab73a9f 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-02 02:54:07,963 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-02 02:54:07,963 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42019,1682996021676' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-02 02:54:07,963 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-02 02:54:07,964 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:54:07,964 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-02 02:54:07,964 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-02 02:54:07,964 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38925,1682996021633' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-02 02:54:07,965 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,966 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,966 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,966 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:54:07,966 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:07,966 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:54:07,976 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/.tmp/cf1/609652eb416344f8b705e1e96171ce1e 2023-05-02 02:54:07,987 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/.tmp/cf2/8243f44fc750458da547c0947dff42a0 2023-05-02 02:54:07,993 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/.tmp/cf1/609652eb416344f8b705e1e96171ce1e as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/609652eb416344f8b705e1e96171ce1e 2023-05-02 02:54:07,997 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/609652eb416344f8b705e1e96171ce1e, entries=10, sequenceid=58, filesize=5.0 K 2023-05-02 02:54:07,998 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/.tmp/cf2/8243f44fc750458da547c0947dff42a0 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/8243f44fc750458da547c0947dff42a0 2023-05-02 02:54:08,002 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/8243f44fc750458da547c0947dff42a0, entries=10, sequenceid=58, filesize=5.0 K 2023-05-02 02:54:08,003 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 8af7d4111b21c70efea08f7beab73a9f in 40ms, sequenceid=58, compaction requested=true 2023-05-02 02:54:08,003 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8af7d4111b21c70efea08f7beab73a9f: 2023-05-02 02:54:08,003 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1682996046681.8af7d4111b21c70efea08f7beab73a9f. 2023-05-02 02:54:08,003 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-02 02:54:08,003 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-02 02:54:08,003 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,003 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-02 02:54:08,003 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-02 02:54:08,005 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,005 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:54:08,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:54:08,005 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,005 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:54:08,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:54:08,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:54:08,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:08,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:08,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:54:08,007 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-05-02 02:54:08,007 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@661f485b[Count = 0] remaining members to acquire global barrier 2023-05-02 02:54:08,007 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-05-02 02:54:08,007 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,009 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,009 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,009 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,009 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,009 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,009 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-02 02:54:08,009 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,009 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-02 02:54:08,010 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-02 02:54:08,010 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-02 02:54:08,009 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-02 02:54:08,009 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,010 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,42019,1682996021676' in zk 2023-05-02 02:54:08,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,010 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-02 02:54:08,010 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-02 02:54:08,010 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,38925,1682996021633' in zk 2023-05-02 02:54:08,010 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,39081,1682996021579' in zk 2023-05-02 02:54:08,011 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-02 02:54:08,011 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:54:08,011 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-02 02:54:08,011 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-02 02:54:08,011 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-02 02:54:08,012 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,012 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:54:08,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:54:08,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:54:08,012 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:54:08,012 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-02 02:54:08,012 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-02 02:54:08,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:54:08,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:54:08,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:08,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:08,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:54:08,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:08,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:08,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,39081,1682996021579': 2023-05-02 02:54:08,015 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-05-02 02:54:08,015 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-05-02 02:54:08,015 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-02 02:54:08,015 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-02 02:54:08,015 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,015 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-02 02:54:08,016 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,016 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:54:08,016 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,017 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,016 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:54:08,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:54:08,017 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,017 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:54:08,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,017 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,017 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:54:08,017 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:54:08,017 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:08,017 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:54:08,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:54:08,018 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:08,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,018 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:54:08,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:08,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:08,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,020 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:08,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:54:08,020 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:08,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,021 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:08,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:08,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,024 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:54:08,024 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:54:08,024 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:54:08,024 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:54:08,024 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:54:08,024 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter because node does not exist (not an error) 2023-05-02 02:54:08,024 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter because node does not exist (not an error) 2023-05-02 02:54:08,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:54:08,024 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,025 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:54:08,025 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:54:08,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:54:08,025 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-05-02 02:54:08,025 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-02 02:54:08,025 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:54:08,025 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:54:08,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:54:08,025 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-05-02 02:54:08,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:54:08,025 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:08,025 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-02 02:54:08,025 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:54:08,025 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:08,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:54:08,026 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:54:08,026 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:54:08,026 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,026 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,026 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:08,026 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:08,026 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,026 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,126 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-02 02:54:08,127 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-02 02:54:08,140 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-02 02:54:08,141 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-05-02 02:54:08,141 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,141 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-05-02 02:54:08,141 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:54:08,141 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-05-02 02:54:08,141 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-05-02 02:54:08,142 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,142 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,145 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:54:08,145 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:54:08,145 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:54:08,145 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:54:08,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:54:08,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:54:08,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:54:08,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:54:08,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:54:08,146 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,146 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-02 02:54:08,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,146 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,146 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,147 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:54:08,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,147 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,147 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-02 02:54:08,147 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:54:08,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,147 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-02 02:54:08,147 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:54:08,147 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:54:08,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,147 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-02 02:54:08,148 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,148 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-02 02:54:08,148 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,148 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-02 02:54:08,148 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-02 02:54:08,148 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-02 02:54:08,148 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:54:08,148 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-02 02:54:08,148 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-02 02:54:08,148 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-02 02:54:08,148 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1682996046681.8af7d4111b21c70efea08f7beab73a9f. 2023-05-02 02:54:08,148 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38925,1682996021633' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-02 02:54:08,148 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,148 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1682996046681.8af7d4111b21c70efea08f7beab73a9f. started... 2023-05-02 02:54:08,148 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-02 02:54:08,148 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:54:08,148 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-02 02:54:08,148 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-02 02:54:08,148 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42019,1682996021676' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-02 02:54:08,148 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8af7d4111b21c70efea08f7beab73a9f 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-02 02:54:08,150 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,150 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,150 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,150 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:54:08,150 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,150 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:54:08,167 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/.tmp/cf1/40f1a5f529284647a6cf211a61001756 2023-05-02 02:54:08,183 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/.tmp/cf2/378b3c3bb4df4380b5f3244397d9fcb2 2023-05-02 02:54:08,188 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/.tmp/cf1/40f1a5f529284647a6cf211a61001756 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/40f1a5f529284647a6cf211a61001756 2023-05-02 02:54:08,192 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/40f1a5f529284647a6cf211a61001756, entries=10, sequenceid=71, filesize=5.0 K 2023-05-02 02:54:08,193 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/.tmp/cf2/378b3c3bb4df4380b5f3244397d9fcb2 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/378b3c3bb4df4380b5f3244397d9fcb2 2023-05-02 02:54:08,196 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/378b3c3bb4df4380b5f3244397d9fcb2, entries=10, sequenceid=71, filesize=5.0 K 2023-05-02 02:54:08,197 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 8af7d4111b21c70efea08f7beab73a9f in 49ms, sequenceid=71, compaction requested=true 2023-05-02 02:54:08,197 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8af7d4111b21c70efea08f7beab73a9f: 2023-05-02 02:54:08,197 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1682996046681.8af7d4111b21c70efea08f7beab73a9f. 2023-05-02 02:54:08,197 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-02 02:54:08,197 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-02 02:54:08,197 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,197 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-02 02:54:08,197 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-02 02:54:08,200 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,200 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:54:08,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:54:08,200 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,200 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:54:08,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:54:08,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:54:08,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:08,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:08,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:54:08,202 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-05-02 02:54:08,202 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4ea87d5b[Count = 0] remaining members to acquire global barrier 2023-05-02 02:54:08,202 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-05-02 02:54:08,202 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,203 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,203 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,203 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,203 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,203 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,203 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,203 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-02 02:54:08,203 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-02 02:54:08,204 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,39081,1682996021579' in zk 2023-05-02 02:54:08,203 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-02 02:54:08,203 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,204 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-02 02:54:08,204 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-02 02:54:08,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,204 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,38925,1682996021633' in zk 2023-05-02 02:54:08,204 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-02 02:54:08,204 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-02 02:54:08,204 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,42019,1682996021676' in zk 2023-05-02 02:54:08,205 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,205 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-02 02:54:08,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:54:08,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:54:08,206 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:54:08,206 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-02 02:54:08,206 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-02 02:54:08,206 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-02 02:54:08,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:54:08,206 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:54:08,206 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:54:08,207 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-02 02:54:08,207 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-02 02:54:08,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:54:08,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:08,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:08,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:54:08,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:08,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:08,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,39081,1682996021579': 2023-05-02 02:54:08,210 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-05-02 02:54:08,210 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-05-02 02:54:08,210 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-02 02:54:08,210 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-02 02:54:08,210 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,210 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-02 02:54:08,212 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,212 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,212 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,212 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,212 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:54:08,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,212 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:54:08,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:54:08,212 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:08,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:54:08,212 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:54:08,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,213 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:08,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:54:08,213 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:54:08,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:54:08,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:54:08,213 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:54:08,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:54:08,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:54:08,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:54:08,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,214 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:08,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:08,215 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:08,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:08,215 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:54:08,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:08,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:08,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,218 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:54:08,218 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,218 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:54:08,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:54:08,218 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,218 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-05-02 02:54:08,218 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-02 02:54:08,218 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:54:08,218 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:54:08,218 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:54:08,218 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:54:08,218 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:54:08,218 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:08,218 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:54:08,218 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:54:08,218 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:54:08,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:54:08,218 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-05-02 02:54:08,218 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:08,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:54:08,218 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:54:08,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:54:08,219 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,219 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-02 02:54:08,219 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,219 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,219 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:08,219 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:08,219 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:54:08,219 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:54:08,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:54:08,219 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:54:08,219 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,319 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-02 02:54:08,320 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-02 02:54:08,332 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-02 02:54:08,334 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-05-02 02:54:08,334 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,334 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-05-02 02:54:08,334 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:54:08,334 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-05-02 02:54:08,334 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-05-02 02:54:08,335 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,335 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,336 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,336 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:54:08,336 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:54:08,336 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:54:08,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:54:08,336 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:54:08,336 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:54:08,336 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:54:08,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:54:08,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:54:08,336 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,336 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-02 02:54:08,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,337 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,337 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,337 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:54:08,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,337 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:54:08,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,337 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:54:08,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,337 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-02 02:54:08,337 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,337 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-02 02:54:08,337 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,337 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,337 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-02 02:54:08,337 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-02 02:54:08,337 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-02 02:54:08,338 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:54:08,337 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:54:08,338 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-02 02:54:08,338 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-02 02:54:08,338 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,338 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-02 02:54:08,338 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-02 02:54:08,338 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42019,1682996021676' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-02 02:54:08,338 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-02 02:54:08,338 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-02 02:54:08,338 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:54:08,338 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1682996046681.8af7d4111b21c70efea08f7beab73a9f. 2023-05-02 02:54:08,338 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-02 02:54:08,338 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-02 02:54:08,338 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38925,1682996021633' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-02 02:54:08,338 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1682996046681.8af7d4111b21c70efea08f7beab73a9f. started... 2023-05-02 02:54:08,338 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8af7d4111b21c70efea08f7beab73a9f 2/2 column families, dataSize=530 B heapSize=2.33 KB 2023-05-02 02:54:08,339 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,341 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,341 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,341 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:54:08,341 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,341 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:54:08,349 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/.tmp/cf1/8db73eea04c64e878c8c4529b89946df 2023-05-02 02:54:08,353 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8db73eea04c64e878c8c4529b89946df 2023-05-02 02:54:08,362 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/.tmp/cf2/a02bf38bab7c419c8ba5d1fb45d02f4c 2023-05-02 02:54:08,365 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a02bf38bab7c419c8ba5d1fb45d02f4c 2023-05-02 02:54:08,366 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/.tmp/cf1/8db73eea04c64e878c8c4529b89946df as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/8db73eea04c64e878c8c4529b89946df 2023-05-02 02:54:08,370 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8db73eea04c64e878c8c4529b89946df 2023-05-02 02:54:08,370 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/8db73eea04c64e878c8c4529b89946df, entries=9, sequenceid=83, filesize=5.1 K 2023-05-02 02:54:08,370 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/.tmp/cf2/a02bf38bab7c419c8ba5d1fb45d02f4c as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/a02bf38bab7c419c8ba5d1fb45d02f4c 2023-05-02 02:54:08,374 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a02bf38bab7c419c8ba5d1fb45d02f4c 2023-05-02 02:54:08,374 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/a02bf38bab7c419c8ba5d1fb45d02f4c, entries=9, sequenceid=83, filesize=5.1 K 2023-05-02 02:54:08,375 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~530 B/530, heapSize ~2.30 KB/2352, currentSize=0 B/0 for 8af7d4111b21c70efea08f7beab73a9f in 37ms, sequenceid=83, compaction requested=true 2023-05-02 02:54:08,375 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8af7d4111b21c70efea08f7beab73a9f: 2023-05-02 02:54:08,375 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1682996046681.8af7d4111b21c70efea08f7beab73a9f. 2023-05-02 02:54:08,375 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-02 02:54:08,375 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-02 02:54:08,375 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,375 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-02 02:54:08,375 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-02 02:54:08,376 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,377 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:54:08,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:54:08,377 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,377 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:54:08,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:54:08,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:54:08,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:08,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:08,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:54:08,379 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-05-02 02:54:08,379 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1ffdf1a9[Count = 0] remaining members to acquire global barrier 2023-05-02 02:54:08,379 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-05-02 02:54:08,379 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,380 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,380 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,380 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,380 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,380 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,380 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-02 02:54:08,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,380 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,380 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-02 02:54:08,380 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,380 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-02 02:54:08,380 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-02 02:54:08,380 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-02 02:54:08,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,380 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,38925,1682996021633' in zk 2023-05-02 02:54:08,380 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-02 02:54:08,380 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-02 02:54:08,380 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,39081,1682996021579' in zk 2023-05-02 02:54:08,380 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,42019,1682996021676' in zk 2023-05-02 02:54:08,385 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-02 02:54:08,386 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,386 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-02 02:54:08,386 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:54:08,386 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:54:08,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,386 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-02 02:54:08,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:54:08,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:54:08,386 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-02 02:54:08,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:54:08,386 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-02 02:54:08,386 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:54:08,386 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-02 02:54:08,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:54:08,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:08,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:08,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:54:08,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:08,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:08,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,39081,1682996021579': 2023-05-02 02:54:08,390 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-05-02 02:54:08,390 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-05-02 02:54:08,390 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-02 02:54:08,390 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-02 02:54:08,390 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,390 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-02 02:54:08,392 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,392 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,392 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:54:08,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:54:08,392 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,392 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:54:08,392 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,392 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,392 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:54:08,392 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:54:08,392 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:54:08,392 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:54:08,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,393 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:08,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:54:08,393 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:54:08,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:54:08,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,393 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:54:08,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:54:08,393 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:08,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:54:08,393 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:08,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:08,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:54:08,395 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:08,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,395 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:08,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:08,395 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:08,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,398 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:54:08,398 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:54:08,398 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:54:08,398 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,398 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:54:08,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:54:08,398 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:54:08,398 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-05-02 02:54:08,398 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-02 02:54:08,398 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:54:08,398 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:54:08,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:54:08,398 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:54:08,398 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:54:08,398 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-05-02 02:54:08,398 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,398 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:54:08,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:54:08,398 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:54:08,399 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:54:08,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:54:08,399 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:08,399 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:08,399 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,399 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,399 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,399 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:08,399 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:08,399 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,399 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:54:08,399 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-02 02:54:08,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:54:08,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:54:08,399 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,499 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-02 02:54:08,500 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-02 02:54:08,508 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-02 02:54:08,509 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-05-02 02:54:08,509 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,509 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-05-02 02:54:08,509 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:54:08,509 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-05-02 02:54:08,509 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-05-02 02:54:08,510 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,510 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,516 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:54:08,516 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:54:08,516 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:54:08,516 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:54:08,516 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:54:08,516 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,516 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:54:08,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:54:08,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:54:08,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:54:08,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,517 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,517 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-02 02:54:08,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,517 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,517 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,517 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:54:08,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,517 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:54:08,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,518 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:54:08,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,518 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,518 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-02 02:54:08,518 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,518 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-02 02:54:08,518 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-02 02:54:08,518 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,518 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-02 02:54:08,518 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-02 02:54:08,518 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:54:08,518 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,518 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:54:08,518 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-02 02:54:08,518 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:54:08,518 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-02 02:54:08,518 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-02 02:54:08,518 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-02 02:54:08,518 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-02 02:54:08,518 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38925,1682996021633' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-02 02:54:08,518 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-02 02:54:08,518 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42019,1682996021676' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-02 02:54:08,518 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-02 02:54:08,518 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-02 02:54:08,519 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1682996046681.8af7d4111b21c70efea08f7beab73a9f. 2023-05-02 02:54:08,519 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1682996046681.8af7d4111b21c70efea08f7beab73a9f. started... 2023-05-02 02:54:08,519 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8af7d4111b21c70efea08f7beab73a9f 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-05-02 02:54:08,520 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,520 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,520 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,520 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,520 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:54:08,520 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:54:08,528 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/.tmp/cf1/d75d441bf5874fe6b71adefb02039137 2023-05-02 02:54:08,532 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d75d441bf5874fe6b71adefb02039137 2023-05-02 02:54:08,542 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/.tmp/cf2/eb4bc94f1cf94bc080443cc926ed2a17 2023-05-02 02:54:08,545 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for eb4bc94f1cf94bc080443cc926ed2a17 2023-05-02 02:54:08,546 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/.tmp/cf1/d75d441bf5874fe6b71adefb02039137 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/d75d441bf5874fe6b71adefb02039137 2023-05-02 02:54:08,550 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d75d441bf5874fe6b71adefb02039137 2023-05-02 02:54:08,550 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/d75d441bf5874fe6b71adefb02039137, entries=5, sequenceid=91, filesize=5.0 K 2023-05-02 02:54:08,550 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/.tmp/cf2/eb4bc94f1cf94bc080443cc926ed2a17 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/eb4bc94f1cf94bc080443cc926ed2a17 2023-05-02 02:54:08,554 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for eb4bc94f1cf94bc080443cc926ed2a17 2023-05-02 02:54:08,554 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/eb4bc94f1cf94bc080443cc926ed2a17, entries=5, sequenceid=91, filesize=5.0 K 2023-05-02 02:54:08,555 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for 8af7d4111b21c70efea08f7beab73a9f in 36ms, sequenceid=91, compaction requested=true 2023-05-02 02:54:08,555 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8af7d4111b21c70efea08f7beab73a9f: 2023-05-02 02:54:08,555 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1682996046681.8af7d4111b21c70efea08f7beab73a9f. 2023-05-02 02:54:08,555 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-02 02:54:08,555 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-02 02:54:08,555 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,555 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-02 02:54:08,555 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-02 02:54:08,556 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,556 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:54:08,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:54:08,557 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,557 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:54:08,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:54:08,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:54:08,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:08,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:08,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:54:08,558 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-05-02 02:54:08,558 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3b9153ac[Count = 0] remaining members to acquire global barrier 2023-05-02 02:54:08,558 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-05-02 02:54:08,558 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,559 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,559 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,559 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,559 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,559 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,560 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,560 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-02 02:54:08,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,560 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-02 02:54:08,560 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,560 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-02 02:54:08,560 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-02 02:54:08,560 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-02 02:54:08,560 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,39081,1682996021579' in zk 2023-05-02 02:54:08,560 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,38925,1682996021633' in zk 2023-05-02 02:54:08,560 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-02 02:54:08,560 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-02 02:54:08,560 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,42019,1682996021676' in zk 2023-05-02 02:54:08,562 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-02 02:54:08,562 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-02 02:54:08,562 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:54:08,562 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-02 02:54:08,562 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-02 02:54:08,562 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:54:08,562 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:54:08,562 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-02 02:54:08,562 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,562 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-02 02:54:08,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:54:08,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:54:08,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:54:08,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:54:08,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:08,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:08,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:54:08,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:08,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:08,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,39081,1682996021579': 2023-05-02 02:54:08,566 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-05-02 02:54:08,566 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-05-02 02:54:08,566 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-02 02:54:08,566 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-02 02:54:08,566 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,566 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-02 02:54:08,567 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,567 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,567 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:54:08,567 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:54:08,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:54:08,567 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,567 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,567 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,567 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,567 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:54:08,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:54:08,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,567 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:54:08,568 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:54:08,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:54:08,568 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:08,568 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:54:08,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:54:08,568 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:54:08,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:54:08,568 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:08,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,569 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:54:08,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:08,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:08,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,570 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:08,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:54:08,570 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:08,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,570 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:08,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:08,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,575 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,575 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,575 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:08,575 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:08,575 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:54:08,575 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:54:08,575 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:54:08,576 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:54:08,576 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-05-02 02:54:08,576 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-02 02:54:08,576 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:54:08,576 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:54:08,576 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:54:08,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:54:08,575 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:54:08,576 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,576 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:54:08,576 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:54:08,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:54:08,576 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:54:08,576 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-05-02 02:54:08,576 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,576 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:54:08,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:54:08,576 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-02 02:54:08,576 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:54:08,576 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:54:08,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:54:08,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:54:08,576 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:08,577 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:08,577 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,577 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,677 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-02 02:54:08,678 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-02 02:54:08,686 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-02 02:54:08,687 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-05-02 02:54:08,687 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,687 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-05-02 02:54:08,687 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:54:08,688 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-05-02 02:54:08,688 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-05-02 02:54:08,688 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,688 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,691 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:54:08,691 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:54:08,691 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:54:08,691 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:54:08,691 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,691 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:54:08,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:54:08,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:54:08,691 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:54:08,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:54:08,691 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,691 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-02 02:54:08,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,692 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,692 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,692 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:54:08,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:54:08,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:54:08,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,692 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,692 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,692 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,693 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-02 02:54:08,693 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,693 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-02 02:54:08,693 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-02 02:54:08,693 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:54:08,693 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-02 02:54:08,693 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,693 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,693 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-02 02:54:08,693 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-02 02:54:08,693 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:54:08,693 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-02 02:54:08,693 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-02 02:54:08,693 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-02 02:54:08,693 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38925,1682996021633' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-02 02:54:08,693 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:54:08,693 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-02 02:54:08,693 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-02 02:54:08,693 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42019,1682996021676' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-02 02:54:08,693 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-02 02:54:08,693 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-02 02:54:08,693 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1682996046681.8af7d4111b21c70efea08f7beab73a9f. 2023-05-02 02:54:08,694 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1682996046681.8af7d4111b21c70efea08f7beab73a9f. started... 2023-05-02 02:54:08,694 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8af7d4111b21c70efea08f7beab73a9f 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-05-02 02:54:08,695 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,695 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,696 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:54:08,696 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,696 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,696 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:54:08,713 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/.tmp/cf1/72ce99472ae84721bfaf8e780648de42 2023-05-02 02:54:08,718 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 72ce99472ae84721bfaf8e780648de42 2023-05-02 02:54:08,730 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/.tmp/cf2/09bd463709ed4f1e83a4a0d35349012e 2023-05-02 02:54:08,734 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 09bd463709ed4f1e83a4a0d35349012e 2023-05-02 02:54:08,735 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/.tmp/cf1/72ce99472ae84721bfaf8e780648de42 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/72ce99472ae84721bfaf8e780648de42 2023-05-02 02:54:08,739 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 72ce99472ae84721bfaf8e780648de42 2023-05-02 02:54:08,739 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/72ce99472ae84721bfaf8e780648de42, entries=5, sequenceid=99, filesize=5.0 K 2023-05-02 02:54:08,740 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/.tmp/cf2/09bd463709ed4f1e83a4a0d35349012e as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/09bd463709ed4f1e83a4a0d35349012e 2023-05-02 02:54:08,744 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 09bd463709ed4f1e83a4a0d35349012e 2023-05-02 02:54:08,744 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/09bd463709ed4f1e83a4a0d35349012e, entries=5, sequenceid=99, filesize=5.0 K 2023-05-02 02:54:08,745 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for 8af7d4111b21c70efea08f7beab73a9f in 51ms, sequenceid=99, compaction requested=true 2023-05-02 02:54:08,745 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8af7d4111b21c70efea08f7beab73a9f: 2023-05-02 02:54:08,745 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1682996046681.8af7d4111b21c70efea08f7beab73a9f. 2023-05-02 02:54:08,745 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-02 02:54:08,745 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-02 02:54:08,745 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,745 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-02 02:54:08,745 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-02 02:54:08,747 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,747 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:54:08,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:54:08,747 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,747 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:54:08,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:54:08,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:54:08,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:08,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:08,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:54:08,749 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-05-02 02:54:08,749 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@27509915[Count = 0] remaining members to acquire global barrier 2023-05-02 02:54:08,749 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-05-02 02:54:08,749 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,752 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,752 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,752 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,752 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,752 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-02 02:54:08,752 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,752 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-02 02:54:08,752 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-02 02:54:08,752 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,752 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,752 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-02 02:54:08,752 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-02 02:54:08,752 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-02 02:54:08,752 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,42019,1682996021676' in zk 2023-05-02 02:54:08,752 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,38925,1682996021633' in zk 2023-05-02 02:54:08,752 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-02 02:54:08,752 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,39081,1682996021579' in zk 2023-05-02 02:54:08,753 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-02 02:54:08,754 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-02 02:54:08,754 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,754 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:54:08,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,754 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-02 02:54:08,754 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:54:08,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:54:08,754 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-02 02:54:08,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:54:08,754 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-02 02:54:08,754 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:54:08,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:54:08,754 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-02 02:54:08,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:54:08,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:08,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:08,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:54:08,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:08,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:08,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,39081,1682996021579': 2023-05-02 02:54:08,757 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-05-02 02:54:08,757 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-05-02 02:54:08,757 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-02 02:54:08,757 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-02 02:54:08,757 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,757 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-02 02:54:08,758 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,758 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,758 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,758 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,758 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:54:08,758 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,758 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,758 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:54:08,759 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:54:08,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:54:08,759 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:54:08,759 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:08,759 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:54:08,759 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:54:08,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:54:08,759 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:54:08,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:54:08,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:54:08,759 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:08,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:54:08,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,760 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:54:08,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:08,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:08,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,761 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:08,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:54:08,761 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:08,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,762 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:08,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:08,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,765 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,765 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:54:08,765 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:54:08,765 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:54:08,765 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:54:08,765 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:54:08,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:54:08,765 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:54:08,765 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-05-02 02:54:08,765 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-02 02:54:08,765 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:54:08,765 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:54:08,765 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:54:08,765 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:54:08,766 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:54:08,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:54:08,765 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:54:08,766 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-05-02 02:54:08,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:54:08,766 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-02 02:54:08,765 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:08,766 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:08,766 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,766 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,766 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,766 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:54:08,766 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:08,766 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:54:08,766 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:54:08,766 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:54:08,766 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:08,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:54:08,866 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-02 02:54:08,867 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-02 02:54:08,869 DEBUG [Listener at localhost/37119] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,870 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(1029): Stored pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=8af7d4111b21c70efea08f7beab73a9f, daughterA=bb895e1df53cda46bfcb5f94fe1710c9, daughterB=024f19513d3a9ded6c060054f1607e68 2023-05-02 02:54:08,871 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=8af7d4111b21c70efea08f7beab73a9f, daughterA=bb895e1df53cda46bfcb5f94fe1710c9, daughterB=024f19513d3a9ded6c060054f1607e68 2023-05-02 02:54:08,871 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=8af7d4111b21c70efea08f7beab73a9f, daughterA=bb895e1df53cda46bfcb5f94fe1710c9, daughterB=024f19513d3a9ded6c060054f1607e68 2023-05-02 02:54:08,871 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=8af7d4111b21c70efea08f7beab73a9f, daughterA=bb895e1df53cda46bfcb5f94fe1710c9, daughterB=024f19513d3a9ded6c060054f1607e68 2023-05-02 02:54:08,873 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=8af7d4111b21c70efea08f7beab73a9f, UNASSIGN}] 2023-05-02 02:54:08,874 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=8af7d4111b21c70efea08f7beab73a9f, UNASSIGN 2023-05-02 02:54:08,875 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=8af7d4111b21c70efea08f7beab73a9f, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:08,875 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1682996046681.8af7d4111b21c70efea08f7beab73a9f.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1682996048874"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1682996048874"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682996048874"}]},"ts":"1682996048874"} 2023-05-02 02:54:08,876 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=74, ppid=73, state=RUNNABLE; CloseRegionProcedure 8af7d4111b21c70efea08f7beab73a9f, server=jenkins-hbase4.apache.org,39081,1682996021579}] 2023-05-02 02:54:09,028 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:09,028 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 8af7d4111b21c70efea08f7beab73a9f, disabling compactions & flushes 2023-05-02 02:54:09,028 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionDeletedRowsDaughter,,1682996046681.8af7d4111b21c70efea08f7beab73a9f. 2023-05-02 02:54:09,028 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1682996046681.8af7d4111b21c70efea08f7beab73a9f. 2023-05-02 02:54:09,028 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1682996046681.8af7d4111b21c70efea08f7beab73a9f. after waiting 0 ms 2023-05-02 02:54:09,028 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1682996046681.8af7d4111b21c70efea08f7beab73a9f. 2023-05-02 02:54:09,028 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 8af7d4111b21c70efea08f7beab73a9f 2/2 column families, dataSize=50 B heapSize=720 B 2023-05-02 02:54:09,038 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/.tmp/cf1/dc659b190e7744abb3e0fe388448fa31 2023-05-02 02:54:09,042 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for dc659b190e7744abb3e0fe388448fa31 2023-05-02 02:54:09,050 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/.tmp/cf2/90a8fcb9bc36402ab2af45c21a1f4254 2023-05-02 02:54:09,054 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 90a8fcb9bc36402ab2af45c21a1f4254 2023-05-02 02:54:09,054 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/.tmp/cf1/dc659b190e7744abb3e0fe388448fa31 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/dc659b190e7744abb3e0fe388448fa31 2023-05-02 02:54:09,058 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for dc659b190e7744abb3e0fe388448fa31 2023-05-02 02:54:09,058 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/dc659b190e7744abb3e0fe388448fa31, entries=1, sequenceid=103, filesize=4.9 K 2023-05-02 02:54:09,059 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/.tmp/cf2/90a8fcb9bc36402ab2af45c21a1f4254 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/90a8fcb9bc36402ab2af45c21a1f4254 2023-05-02 02:54:09,063 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 90a8fcb9bc36402ab2af45c21a1f4254 2023-05-02 02:54:09,063 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/90a8fcb9bc36402ab2af45c21a1f4254, entries=1, sequenceid=103, filesize=4.9 K 2023-05-02 02:54:09,064 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~50 B/50, heapSize ~688 B/688, currentSize=0 B/0 for 8af7d4111b21c70efea08f7beab73a9f in 36ms, sequenceid=103, compaction requested=true 2023-05-02 02:54:09,069 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=1 2023-05-02 02:54:09,069 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-02 02:54:09,069 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionDeletedRowsDaughter,,1682996046681.8af7d4111b21c70efea08f7beab73a9f. 2023-05-02 02:54:09,069 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 8af7d4111b21c70efea08f7beab73a9f: 2023-05-02 02:54:09,071 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:09,071 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=8af7d4111b21c70efea08f7beab73a9f, regionState=CLOSED 2023-05-02 02:54:09,071 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1682996046681.8af7d4111b21c70efea08f7beab73a9f.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1682996049071"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682996049071"}]},"ts":"1682996049071"} 2023-05-02 02:54:09,074 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=74, resume processing ppid=73 2023-05-02 02:54:09,074 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=74, ppid=73, state=SUCCESS; CloseRegionProcedure 8af7d4111b21c70efea08f7beab73a9f, server=jenkins-hbase4.apache.org,39081,1682996021579 in 197 msec 2023-05-02 02:54:09,075 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=73, resume processing ppid=72 2023-05-02 02:54:09,075 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=73, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=8af7d4111b21c70efea08f7beab73a9f, UNASSIGN in 201 msec 2023-05-02 02:54:09,090 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=72 splitting 20 storefiles, region=8af7d4111b21c70efea08f7beab73a9f, threads=16 2023-05-02 02:54:09,091 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/09bd463709ed4f1e83a4a0d35349012e for region: 8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:09,092 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/378b3c3bb4df4380b5f3244397d9fcb2 for region: 8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:09,092 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/76b4e49972d544bdb7a8702299d57cc6 for region: 8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:09,096 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/8243f44fc750458da547c0947dff42a0 for region: 8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:09,098 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/90a8fcb9bc36402ab2af45c21a1f4254 for region: 8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:09,098 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/a02bf38bab7c419c8ba5d1fb45d02f4c for region: 8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:09,099 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/e6bbd81d56884f75a2ab34f638d3dc7f for region: 8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:09,099 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/eb4bc94f1cf94bc080443cc926ed2a17 for region: 8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:09,099 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/eecb4f6bc8094e669b2ee7981bc5a39f for region: 8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:09,099 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/f894abdeb02b4e87bc8789e67b483cc0 for region: 8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:09,103 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/3189d0758d394d4290511c7d95e297bb for region: 8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:09,103 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/40f1a5f529284647a6cf211a61001756 for region: 8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:09,104 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/609652eb416344f8b705e1e96171ce1e for region: 8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:09,104 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/72ce99472ae84721bfaf8e780648de42 for region: 8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:09,104 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/8db73eea04c64e878c8c4529b89946df for region: 8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:09,104 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/b0c982e0384d423aa455befed4ca21c3 for region: 8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:09,107 INFO [StoreFileSplitter-pool-4] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 90a8fcb9bc36402ab2af45c21a1f4254 2023-05-02 02:54:09,110 INFO [StoreFileSplitter-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 09bd463709ed4f1e83a4a0d35349012e 2023-05-02 02:54:09,124 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/f894abdeb02b4e87bc8789e67b483cc0, top=false 2023-05-02 02:54:09,124 DEBUG [StoreFileSplitter-pool-15] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/b0c982e0384d423aa455befed4ca21c3, top=false 2023-05-02 02:54:09,125 INFO [StoreFileSplitter-pool-7] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for eb4bc94f1cf94bc080443cc926ed2a17 2023-05-02 02:54:09,126 INFO [StoreFileSplitter-pool-4] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 90a8fcb9bc36402ab2af45c21a1f4254 2023-05-02 02:54:09,126 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/90a8fcb9bc36402ab2af45c21a1f4254, top=true 2023-05-02 02:54:09,126 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/40f1a5f529284647a6cf211a61001756, top=false 2023-05-02 02:54:09,126 INFO [StoreFileSplitter-pool-14] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8db73eea04c64e878c8c4529b89946df 2023-05-02 02:54:09,128 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/8243f44fc750458da547c0947dff42a0, top=false 2023-05-02 02:54:09,128 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/609652eb416344f8b705e1e96171ce1e, top=false 2023-05-02 02:54:09,129 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/378b3c3bb4df4380b5f3244397d9fcb2, top=false 2023-05-02 02:54:09,131 INFO [StoreFileSplitter-pool-13] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 72ce99472ae84721bfaf8e780648de42 2023-05-02 02:54:09,132 INFO [StoreFileSplitter-pool-5] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a02bf38bab7c419c8ba5d1fb45d02f4c 2023-05-02 02:54:09,136 INFO [StoreFileSplitter-pool-13] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 72ce99472ae84721bfaf8e780648de42 2023-05-02 02:54:09,136 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/72ce99472ae84721bfaf8e780648de42, top=true 2023-05-02 02:54:09,145 INFO [StoreFileSplitter-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 09bd463709ed4f1e83a4a0d35349012e 2023-05-02 02:54:09,145 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/09bd463709ed4f1e83a4a0d35349012e, top=true 2023-05-02 02:54:09,146 INFO [StoreFileSplitter-pool-7] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for eb4bc94f1cf94bc080443cc926ed2a17 2023-05-02 02:54:09,146 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/eb4bc94f1cf94bc080443cc926ed2a17, top=true 2023-05-02 02:54:09,162 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/.splits/bb895e1df53cda46bfcb5f94fe1710c9/cf1/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-40f1a5f529284647a6cf211a61001756 for child: bb895e1df53cda46bfcb5f94fe1710c9, parent: 8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:09,162 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/.splits/bb895e1df53cda46bfcb5f94fe1710c9/cf2/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-378b3c3bb4df4380b5f3244397d9fcb2 for child: bb895e1df53cda46bfcb5f94fe1710c9, parent: 8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:09,162 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/.splits/bb895e1df53cda46bfcb5f94fe1710c9/cf2/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-8243f44fc750458da547c0947dff42a0 for child: bb895e1df53cda46bfcb5f94fe1710c9, parent: 8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:09,163 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/.splits/024f19513d3a9ded6c060054f1607e68/cf2/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-90a8fcb9bc36402ab2af45c21a1f4254 for child: 024f19513d3a9ded6c060054f1607e68, parent: 8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:09,163 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/90a8fcb9bc36402ab2af45c21a1f4254 for region: 8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:09,163 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/d75d441bf5874fe6b71adefb02039137 for region: 8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:09,172 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/.splits/bb895e1df53cda46bfcb5f94fe1710c9/cf1/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-609652eb416344f8b705e1e96171ce1e for child: bb895e1df53cda46bfcb5f94fe1710c9, parent: 8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:09,172 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/.splits/bb895e1df53cda46bfcb5f94fe1710c9/cf2/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-f894abdeb02b4e87bc8789e67b483cc0 for child: bb895e1df53cda46bfcb5f94fe1710c9, parent: 8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:09,176 INFO [StoreFileSplitter-pool-15] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/.splits/bb895e1df53cda46bfcb5f94fe1710c9/cf1/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-b0c982e0384d423aa455befed4ca21c3 for child: bb895e1df53cda46bfcb5f94fe1710c9, parent: 8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:09,176 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/.splits/024f19513d3a9ded6c060054f1607e68/cf1/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-72ce99472ae84721bfaf8e780648de42 for child: 024f19513d3a9ded6c060054f1607e68, parent: 8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:09,176 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/72ce99472ae84721bfaf8e780648de42 for region: 8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:09,176 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/dc659b190e7744abb3e0fe388448fa31 for region: 8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:09,179 INFO [StoreFileSplitter-pool-4] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d75d441bf5874fe6b71adefb02039137 2023-05-02 02:54:09,180 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/8243f44fc750458da547c0947dff42a0 for region: 8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:09,180 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/f6fbf066cf0a4af6b2ea76697cc904da for region: 8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:09,181 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/40f1a5f529284647a6cf211a61001756 for region: 8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:09,181 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/fa00f661973547868b8b686731148b94 for region: 8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:09,184 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/f894abdeb02b4e87bc8789e67b483cc0 for region: 8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:09,185 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/b0c982e0384d423aa455befed4ca21c3 for region: 8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:09,187 INFO [StoreFileSplitter-pool-4] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d75d441bf5874fe6b71adefb02039137 2023-05-02 02:54:09,187 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/d75d441bf5874fe6b71adefb02039137, top=true 2023-05-02 02:54:09,189 INFO [StoreFileSplitter-pool-5] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a02bf38bab7c419c8ba5d1fb45d02f4c 2023-05-02 02:54:09,198 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/.splits/024f19513d3a9ded6c060054f1607e68/cf2/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-eb4bc94f1cf94bc080443cc926ed2a17 for child: 024f19513d3a9ded6c060054f1607e68, parent: 8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:09,198 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/eb4bc94f1cf94bc080443cc926ed2a17 for region: 8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:09,217 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/609652eb416344f8b705e1e96171ce1e for region: 8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:09,219 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/.splits/024f19513d3a9ded6c060054f1607e68/cf2/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-09bd463709ed4f1e83a4a0d35349012e for child: 024f19513d3a9ded6c060054f1607e68, parent: 8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:09,219 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/09bd463709ed4f1e83a4a0d35349012e for region: 8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:09,226 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/.splits/024f19513d3a9ded6c060054f1607e68/cf1/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-d75d441bf5874fe6b71adefb02039137 for child: 024f19513d3a9ded6c060054f1607e68, parent: 8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:09,226 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/d75d441bf5874fe6b71adefb02039137 for region: 8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:09,226 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/76b4e49972d544bdb7a8702299d57cc6 for region: 8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:09,237 INFO [StoreFileSplitter-pool-13] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for dc659b190e7744abb3e0fe388448fa31 2023-05-02 02:54:09,241 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/378b3c3bb4df4380b5f3244397d9fcb2 for region: 8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:09,242 INFO [StoreFileSplitter-pool-13] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for dc659b190e7744abb3e0fe388448fa31 2023-05-02 02:54:09,243 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/dc659b190e7744abb3e0fe388448fa31, top=true 2023-05-02 02:54:09,248 INFO [StoreFileSplitter-pool-14] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8db73eea04c64e878c8c4529b89946df 2023-05-02 02:54:09,260 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/a02bf38bab7c419c8ba5d1fb45d02f4c for region: 8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:09,279 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/3189d0758d394d4290511c7d95e297bb for region: 8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:09,279 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/e6bbd81d56884f75a2ab34f638d3dc7f for region: 8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:09,284 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/.splits/024f19513d3a9ded6c060054f1607e68/cf1/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-dc659b190e7744abb3e0fe388448fa31 for child: 024f19513d3a9ded6c060054f1607e68, parent: 8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:09,284 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/8db73eea04c64e878c8c4529b89946df for region: 8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:09,284 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/dc659b190e7744abb3e0fe388448fa31 for region: 8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:09,288 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/eecb4f6bc8094e669b2ee7981bc5a39f for region: 8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:09,297 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/f6fbf066cf0a4af6b2ea76697cc904da for region: 8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:09,697 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/fa00f661973547868b8b686731148b94 for region: 8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:09,697 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=72 split storefiles for region 8af7d4111b21c70efea08f7beab73a9f Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-05-02 02:54:09,735 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-05-02 02:54:09,737 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-05-02 02:54:09,739 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1682996046681.8af7d4111b21c70efea08f7beab73a9f.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1682996049739"},{"qualifier":"splitA","vlen":75,"tag":[],"timestamp":"1682996049739"},{"qualifier":"splitB","vlen":75,"tag":[],"timestamp":"1682996049739"}]},"ts":"1682996049739"} 2023-05-02 02:54:09,739 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1682996048869.bb895e1df53cda46bfcb5f94fe1710c9.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1682996049739"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682996049739"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682996049739"}]},"ts":"1682996049739"} 2023-05-02 02:54:09,739 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1682996048869.024f19513d3a9ded6c060054f1607e68.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1682996049739"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682996049739"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682996049739"}]},"ts":"1682996049739"} 2023-05-02 02:54:09,742 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=bb895e1df53cda46bfcb5f94fe1710c9, ASSIGN}, {pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=024f19513d3a9ded6c060054f1607e68, ASSIGN}] 2023-05-02 02:54:09,743 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=024f19513d3a9ded6c060054f1607e68, ASSIGN 2023-05-02 02:54:09,743 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=024f19513d3a9ded6c060054f1607e68, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,39081,1682996021579; forceNewPlan=false, retain=false 2023-05-02 02:54:09,744 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=bb895e1df53cda46bfcb5f94fe1710c9, ASSIGN 2023-05-02 02:54:09,745 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=bb895e1df53cda46bfcb5f94fe1710c9, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,39081,1682996021579; forceNewPlan=false, retain=false 2023-05-02 02:54:09,826 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-05-02 02:54:09,894 INFO [jenkins-hbase4:41119] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-05-02 02:54:09,895 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=bb895e1df53cda46bfcb5f94fe1710c9, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:09,896 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1682996048869.bb895e1df53cda46bfcb5f94fe1710c9.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1682996049895"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1682996049895"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682996049895"}]},"ts":"1682996049895"} 2023-05-02 02:54:09,897 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=77, ppid=75, state=RUNNABLE; OpenRegionProcedure bb895e1df53cda46bfcb5f94fe1710c9, server=jenkins-hbase4.apache.org,39081,1682996021579}] 2023-05-02 02:54:09,900 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=024f19513d3a9ded6c060054f1607e68, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:09,900 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1682996048869.024f19513d3a9ded6c060054f1607e68.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1682996049900"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1682996049900"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682996049900"}]},"ts":"1682996049900"} 2023-05-02 02:54:09,901 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=78, ppid=76, state=RUNNABLE; OpenRegionProcedure 024f19513d3a9ded6c060054f1607e68, server=jenkins-hbase4.apache.org,39081,1682996021579}] 2023-05-02 02:54:10,052 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionDeletedRowsDaughter,60,1682996048869.024f19513d3a9ded6c060054f1607e68. 2023-05-02 02:54:10,052 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 024f19513d3a9ded6c060054f1607e68, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1682996048869.024f19513d3a9ded6c060054f1607e68.', STARTKEY => '60', ENDKEY => ''} 2023-05-02 02:54:10,052 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-02 02:54:10,053 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter 024f19513d3a9ded6c060054f1607e68 2023-05-02 02:54:10,053 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionDeletedRowsDaughter,60,1682996048869.024f19513d3a9ded6c060054f1607e68.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-02 02:54:10,053 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 024f19513d3a9ded6c060054f1607e68 2023-05-02 02:54:10,053 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 024f19513d3a9ded6c060054f1607e68 2023-05-02 02:54:10,054 INFO [StoreOpener-024f19513d3a9ded6c060054f1607e68-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 024f19513d3a9ded6c060054f1607e68 2023-05-02 02:54:10,055 DEBUG [StoreOpener-024f19513d3a9ded6c060054f1607e68-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf1 2023-05-02 02:54:10,055 DEBUG [StoreOpener-024f19513d3a9ded6c060054f1607e68-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf1 2023-05-02 02:54:10,055 INFO [StoreOpener-024f19513d3a9ded6c060054f1607e68-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 024f19513d3a9ded6c060054f1607e68 columnFamilyName cf1 2023-05-02 02:54:10,065 DEBUG [StoreOpener-024f19513d3a9ded6c060054f1607e68-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf1/3189d0758d394d4290511c7d95e297bb.8af7d4111b21c70efea08f7beab73a9f->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/3189d0758d394d4290511c7d95e297bb-top 2023-05-02 02:54:10,069 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8db73eea04c64e878c8c4529b89946df.8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:10,070 DEBUG [StoreOpener-024f19513d3a9ded6c060054f1607e68-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf1/8db73eea04c64e878c8c4529b89946df.8af7d4111b21c70efea08f7beab73a9f->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/8db73eea04c64e878c8c4529b89946df-top 2023-05-02 02:54:10,075 DEBUG [StoreOpener-024f19513d3a9ded6c060054f1607e68-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf1/f6fbf066cf0a4af6b2ea76697cc904da.8af7d4111b21c70efea08f7beab73a9f->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/f6fbf066cf0a4af6b2ea76697cc904da-top 2023-05-02 02:54:10,079 DEBUG [StoreOpener-024f19513d3a9ded6c060054f1607e68-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf1/fa00f661973547868b8b686731148b94.8af7d4111b21c70efea08f7beab73a9f->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/fa00f661973547868b8b686731148b94-top 2023-05-02 02:54:10,083 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 72ce99472ae84721bfaf8e780648de42 2023-05-02 02:54:10,083 DEBUG [StoreOpener-024f19513d3a9ded6c060054f1607e68-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf1/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-72ce99472ae84721bfaf8e780648de42 2023-05-02 02:54:10,087 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d75d441bf5874fe6b71adefb02039137 2023-05-02 02:54:10,087 DEBUG [StoreOpener-024f19513d3a9ded6c060054f1607e68-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf1/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-d75d441bf5874fe6b71adefb02039137 2023-05-02 02:54:10,090 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for dc659b190e7744abb3e0fe388448fa31 2023-05-02 02:54:10,090 DEBUG [StoreOpener-024f19513d3a9ded6c060054f1607e68-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf1/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-dc659b190e7744abb3e0fe388448fa31 2023-05-02 02:54:10,091 INFO [StoreOpener-024f19513d3a9ded6c060054f1607e68-1] regionserver.HStore(310): Store=024f19513d3a9ded6c060054f1607e68/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-02 02:54:10,091 INFO [StoreOpener-024f19513d3a9ded6c060054f1607e68-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 024f19513d3a9ded6c060054f1607e68 2023-05-02 02:54:10,092 DEBUG [StoreOpener-024f19513d3a9ded6c060054f1607e68-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf2 2023-05-02 02:54:10,092 DEBUG [StoreOpener-024f19513d3a9ded6c060054f1607e68-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf2 2023-05-02 02:54:10,092 INFO [StoreOpener-024f19513d3a9ded6c060054f1607e68-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 024f19513d3a9ded6c060054f1607e68 columnFamilyName cf2 2023-05-02 02:54:10,105 DEBUG [StoreOpener-024f19513d3a9ded6c060054f1607e68-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf2/76b4e49972d544bdb7a8702299d57cc6.8af7d4111b21c70efea08f7beab73a9f->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/76b4e49972d544bdb7a8702299d57cc6-top 2023-05-02 02:54:10,109 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a02bf38bab7c419c8ba5d1fb45d02f4c.8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:10,110 DEBUG [StoreOpener-024f19513d3a9ded6c060054f1607e68-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf2/a02bf38bab7c419c8ba5d1fb45d02f4c.8af7d4111b21c70efea08f7beab73a9f->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/a02bf38bab7c419c8ba5d1fb45d02f4c-top 2023-05-02 02:54:10,115 DEBUG [StoreOpener-024f19513d3a9ded6c060054f1607e68-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf2/e6bbd81d56884f75a2ab34f638d3dc7f.8af7d4111b21c70efea08f7beab73a9f->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/e6bbd81d56884f75a2ab34f638d3dc7f-top 2023-05-02 02:54:10,120 DEBUG [StoreOpener-024f19513d3a9ded6c060054f1607e68-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf2/eecb4f6bc8094e669b2ee7981bc5a39f.8af7d4111b21c70efea08f7beab73a9f->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/eecb4f6bc8094e669b2ee7981bc5a39f-top 2023-05-02 02:54:10,123 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 09bd463709ed4f1e83a4a0d35349012e 2023-05-02 02:54:10,123 DEBUG [StoreOpener-024f19513d3a9ded6c060054f1607e68-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf2/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-09bd463709ed4f1e83a4a0d35349012e 2023-05-02 02:54:10,127 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 90a8fcb9bc36402ab2af45c21a1f4254 2023-05-02 02:54:10,127 DEBUG [StoreOpener-024f19513d3a9ded6c060054f1607e68-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf2/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-90a8fcb9bc36402ab2af45c21a1f4254 2023-05-02 02:54:10,130 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for eb4bc94f1cf94bc080443cc926ed2a17 2023-05-02 02:54:10,131 DEBUG [StoreOpener-024f19513d3a9ded6c060054f1607e68-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf2/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-eb4bc94f1cf94bc080443cc926ed2a17 2023-05-02 02:54:10,131 INFO [StoreOpener-024f19513d3a9ded6c060054f1607e68-1] regionserver.HStore(310): Store=024f19513d3a9ded6c060054f1607e68/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-02 02:54:10,132 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68 2023-05-02 02:54:10,133 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68 2023-05-02 02:54:10,134 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-02 02:54:10,135 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 024f19513d3a9ded6c060054f1607e68 2023-05-02 02:54:10,136 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 024f19513d3a9ded6c060054f1607e68; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63252910, jitterRate=-0.057458192110061646}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-02 02:54:10,136 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 024f19513d3a9ded6c060054f1607e68: 2023-05-02 02:54:10,137 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,60,1682996048869.024f19513d3a9ded6c060054f1607e68., pid=78, masterSystemTime=1682996050049 2023-05-02 02:54:10,137 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-02 02:54:10,138 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-05-02 02:54:10,138 DEBUG [RS:0;jenkins-hbase4:39081-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-02 02:54:10,138 DEBUG [RS:0;jenkins-hbase4:39081-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-02 02:54:10,138 INFO [RS:0;jenkins-hbase4:39081-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1682996048869.024f19513d3a9ded6c060054f1607e68. 2023-05-02 02:54:10,138 DEBUG [RS:0;jenkins-hbase4:39081-shortCompactions-0] regionserver.HStore(1912): 024f19513d3a9ded6c060054f1607e68/cf1 is initiating minor compaction (all files) 2023-05-02 02:54:10,138 INFO [RS:0;jenkins-hbase4:39081-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1682996048869.024f19513d3a9ded6c060054f1607e68. 2023-05-02 02:54:10,138 INFO [RS:0;jenkins-hbase4:39081-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 024f19513d3a9ded6c060054f1607e68/cf1 in testSplitTableRegionDeletedRowsDaughter,60,1682996048869.024f19513d3a9ded6c060054f1607e68. 2023-05-02 02:54:10,138 DEBUG [RS:0;jenkins-hbase4:39081-longCompactions-0] regionserver.HStore(1912): 024f19513d3a9ded6c060054f1607e68/cf2 is initiating minor compaction (all files) 2023-05-02 02:54:10,138 INFO [RS:0;jenkins-hbase4:39081-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 024f19513d3a9ded6c060054f1607e68/cf2 in testSplitTableRegionDeletedRowsDaughter,60,1682996048869.024f19513d3a9ded6c060054f1607e68. 2023-05-02 02:54:10,139 INFO [RS:0;jenkins-hbase4:39081-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf1/3189d0758d394d4290511c7d95e297bb.8af7d4111b21c70efea08f7beab73a9f->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/3189d0758d394d4290511c7d95e297bb-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf1/f6fbf066cf0a4af6b2ea76697cc904da.8af7d4111b21c70efea08f7beab73a9f->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/f6fbf066cf0a4af6b2ea76697cc904da-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf1/fa00f661973547868b8b686731148b94.8af7d4111b21c70efea08f7beab73a9f->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/fa00f661973547868b8b686731148b94-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf1/8db73eea04c64e878c8c4529b89946df.8af7d4111b21c70efea08f7beab73a9f->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/8db73eea04c64e878c8c4529b89946df-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf1/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-d75d441bf5874fe6b71adefb02039137, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf1/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-72ce99472ae84721bfaf8e780648de42, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf1/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-dc659b190e7744abb3e0fe388448fa31] into tmpdir=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/.tmp, totalSize=34.8 K 2023-05-02 02:54:10,139 INFO [RS:0;jenkins-hbase4:39081-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf2/76b4e49972d544bdb7a8702299d57cc6.8af7d4111b21c70efea08f7beab73a9f->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/76b4e49972d544bdb7a8702299d57cc6-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf2/e6bbd81d56884f75a2ab34f638d3dc7f.8af7d4111b21c70efea08f7beab73a9f->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/e6bbd81d56884f75a2ab34f638d3dc7f-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf2/eecb4f6bc8094e669b2ee7981bc5a39f.8af7d4111b21c70efea08f7beab73a9f->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/eecb4f6bc8094e669b2ee7981bc5a39f-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf2/a02bf38bab7c419c8ba5d1fb45d02f4c.8af7d4111b21c70efea08f7beab73a9f->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/a02bf38bab7c419c8ba5d1fb45d02f4c-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf2/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-eb4bc94f1cf94bc080443cc926ed2a17, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf2/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-09bd463709ed4f1e83a4a0d35349012e, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf2/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-90a8fcb9bc36402ab2af45c21a1f4254] into tmpdir=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/.tmp, totalSize=34.8 K 2023-05-02 02:54:10,139 DEBUG [RS:0;jenkins-hbase4:39081-shortCompactions-0] compactions.Compactor(207): Compacting 3189d0758d394d4290511c7d95e297bb.8af7d4111b21c70efea08f7beab73a9f, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1682996047185 2023-05-02 02:54:10,139 DEBUG [RS:0;jenkins-hbase4:39081-longCompactions-0] compactions.Compactor(207): Compacting 76b4e49972d544bdb7a8702299d57cc6.8af7d4111b21c70efea08f7beab73a9f, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1682996047185 2023-05-02 02:54:10,140 DEBUG [RS:0;jenkins-hbase4:39081-shortCompactions-0] compactions.Compactor(207): Compacting f6fbf066cf0a4af6b2ea76697cc904da.8af7d4111b21c70efea08f7beab73a9f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1682996047401 2023-05-02 02:54:10,140 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,60,1682996048869.024f19513d3a9ded6c060054f1607e68. 2023-05-02 02:54:10,140 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionDeletedRowsDaughter,60,1682996048869.024f19513d3a9ded6c060054f1607e68. 2023-05-02 02:54:10,140 DEBUG [RS:0;jenkins-hbase4:39081-longCompactions-0] compactions.Compactor(207): Compacting e6bbd81d56884f75a2ab34f638d3dc7f.8af7d4111b21c70efea08f7beab73a9f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1682996047401 2023-05-02 02:54:10,140 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionDeletedRowsDaughter,,1682996048869.bb895e1df53cda46bfcb5f94fe1710c9. 2023-05-02 02:54:10,140 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => bb895e1df53cda46bfcb5f94fe1710c9, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1682996048869.bb895e1df53cda46bfcb5f94fe1710c9.', STARTKEY => '', ENDKEY => '60'} 2023-05-02 02:54:10,140 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=024f19513d3a9ded6c060054f1607e68, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:10,140 DEBUG [RS:0;jenkins-hbase4:39081-longCompactions-0] compactions.Compactor(207): Compacting eecb4f6bc8094e669b2ee7981bc5a39f.8af7d4111b21c70efea08f7beab73a9f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1682996047578 2023-05-02 02:54:10,140 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,60,1682996048869.024f19513d3a9ded6c060054f1607e68.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1682996050140"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1682996050140"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1682996050140"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682996050140"}]},"ts":"1682996050140"} 2023-05-02 02:54:10,141 DEBUG [RS:0;jenkins-hbase4:39081-shortCompactions-0] compactions.Compactor(207): Compacting fa00f661973547868b8b686731148b94.8af7d4111b21c70efea08f7beab73a9f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1682996047578 2023-05-02 02:54:10,141 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-02 02:54:10,141 DEBUG [RS:0;jenkins-hbase4:39081-longCompactions-0] compactions.Compactor(207): Compacting a02bf38bab7c419c8ba5d1fb45d02f4c.8af7d4111b21c70efea08f7beab73a9f, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1682996048321 2023-05-02 02:54:10,141 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter bb895e1df53cda46bfcb5f94fe1710c9 2023-05-02 02:54:10,141 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionDeletedRowsDaughter,,1682996048869.bb895e1df53cda46bfcb5f94fe1710c9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-02 02:54:10,141 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for bb895e1df53cda46bfcb5f94fe1710c9 2023-05-02 02:54:10,141 DEBUG [RS:0;jenkins-hbase4:39081-shortCompactions-0] compactions.Compactor(207): Compacting 8db73eea04c64e878c8c4529b89946df.8af7d4111b21c70efea08f7beab73a9f, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1682996048321 2023-05-02 02:54:10,141 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for bb895e1df53cda46bfcb5f94fe1710c9 2023-05-02 02:54:10,142 DEBUG [RS:0;jenkins-hbase4:39081-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-eb4bc94f1cf94bc080443cc926ed2a17, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-05-02 02:54:10,142 DEBUG [RS:0;jenkins-hbase4:39081-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-d75d441bf5874fe6b71adefb02039137, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-05-02 02:54:10,142 DEBUG [RS:0;jenkins-hbase4:39081-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-09bd463709ed4f1e83a4a0d35349012e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-05-02 02:54:10,142 DEBUG [RS:0;jenkins-hbase4:39081-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-72ce99472ae84721bfaf8e780648de42, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-05-02 02:54:10,142 DEBUG [RS:0;jenkins-hbase4:39081-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-90a8fcb9bc36402ab2af45c21a1f4254, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-05-02 02:54:10,143 DEBUG [RS:0;jenkins-hbase4:39081-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-dc659b190e7744abb3e0fe388448fa31, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-05-02 02:54:10,143 INFO [StoreOpener-bb895e1df53cda46bfcb5f94fe1710c9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region bb895e1df53cda46bfcb5f94fe1710c9 2023-05-02 02:54:10,144 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=78, resume processing ppid=76 2023-05-02 02:54:10,144 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=78, ppid=76, state=SUCCESS; OpenRegionProcedure 024f19513d3a9ded6c060054f1607e68, server=jenkins-hbase4.apache.org,39081,1682996021579 in 241 msec 2023-05-02 02:54:10,144 DEBUG [StoreOpener-bb895e1df53cda46bfcb5f94fe1710c9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf1 2023-05-02 02:54:10,144 DEBUG [StoreOpener-bb895e1df53cda46bfcb5f94fe1710c9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf1 2023-05-02 02:54:10,145 INFO [StoreOpener-bb895e1df53cda46bfcb5f94fe1710c9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region bb895e1df53cda46bfcb5f94fe1710c9 columnFamilyName cf1 2023-05-02 02:54:10,145 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=76, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=024f19513d3a9ded6c060054f1607e68, ASSIGN in 402 msec 2023-05-02 02:54:10,159 DEBUG [StoreOpener-bb895e1df53cda46bfcb5f94fe1710c9-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf1/3189d0758d394d4290511c7d95e297bb.8af7d4111b21c70efea08f7beab73a9f->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/3189d0758d394d4290511c7d95e297bb-bottom 2023-05-02 02:54:10,160 INFO [RS:0;jenkins-hbase4:39081-longCompactions-0] throttle.PressureAwareThroughputController(145): 024f19513d3a9ded6c060054f1607e68#cf2#compaction#72 average throughput is 0.00 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-02 02:54:10,161 INFO [RS:0;jenkins-hbase4:39081-shortCompactions-0] throttle.PressureAwareThroughputController(145): 024f19513d3a9ded6c060054f1607e68#cf1#compaction#73 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-02 02:54:10,163 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8db73eea04c64e878c8c4529b89946df.8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:10,163 DEBUG [StoreOpener-bb895e1df53cda46bfcb5f94fe1710c9-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf1/8db73eea04c64e878c8c4529b89946df.8af7d4111b21c70efea08f7beab73a9f->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/8db73eea04c64e878c8c4529b89946df-bottom 2023-05-02 02:54:10,166 DEBUG [StoreOpener-bb895e1df53cda46bfcb5f94fe1710c9-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf1/f6fbf066cf0a4af6b2ea76697cc904da.8af7d4111b21c70efea08f7beab73a9f->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/f6fbf066cf0a4af6b2ea76697cc904da-bottom 2023-05-02 02:54:10,174 DEBUG [StoreOpener-bb895e1df53cda46bfcb5f94fe1710c9-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf1/fa00f661973547868b8b686731148b94.8af7d4111b21c70efea08f7beab73a9f->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/fa00f661973547868b8b686731148b94-bottom 2023-05-02 02:54:10,177 DEBUG [StoreOpener-bb895e1df53cda46bfcb5f94fe1710c9-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf1/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-40f1a5f529284647a6cf211a61001756 2023-05-02 02:54:10,186 DEBUG [StoreOpener-bb895e1df53cda46bfcb5f94fe1710c9-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf1/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-609652eb416344f8b705e1e96171ce1e 2023-05-02 02:54:10,191 DEBUG [StoreOpener-bb895e1df53cda46bfcb5f94fe1710c9-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf1/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-b0c982e0384d423aa455befed4ca21c3 2023-05-02 02:54:10,191 INFO [StoreOpener-bb895e1df53cda46bfcb5f94fe1710c9-1] regionserver.HStore(310): Store=bb895e1df53cda46bfcb5f94fe1710c9/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-02 02:54:10,191 INFO [StoreOpener-bb895e1df53cda46bfcb5f94fe1710c9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region bb895e1df53cda46bfcb5f94fe1710c9 2023-05-02 02:54:10,193 DEBUG [RS:0;jenkins-hbase4:39081-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 326e4bf0cbf6466c986237da67de478c: NONE, but ROW specified in column family configuration 2023-05-02 02:54:10,193 DEBUG [StoreOpener-bb895e1df53cda46bfcb5f94fe1710c9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf2 2023-05-02 02:54:10,193 DEBUG [StoreOpener-bb895e1df53cda46bfcb5f94fe1710c9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf2 2023-05-02 02:54:10,193 INFO [StoreOpener-bb895e1df53cda46bfcb5f94fe1710c9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region bb895e1df53cda46bfcb5f94fe1710c9 columnFamilyName cf2 2023-05-02 02:54:10,193 DEBUG [RS:0;jenkins-hbase4:39081-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 7d9d593eeacf44998515b23aa082bc6e: NONE, but ROW specified in column family configuration 2023-05-02 02:54:10,194 DEBUG [RS:0;jenkins-hbase4:39081-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/.tmp/cf2/326e4bf0cbf6466c986237da67de478c as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf2/326e4bf0cbf6466c986237da67de478c 2023-05-02 02:54:10,194 DEBUG [RS:0;jenkins-hbase4:39081-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/.tmp/cf1/7d9d593eeacf44998515b23aa082bc6e as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf1/7d9d593eeacf44998515b23aa082bc6e 2023-05-02 02:54:10,198 DEBUG [RS:0;jenkins-hbase4:39081-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 326e4bf0cbf6466c986237da67de478c: NONE, but ROW specified in column family configuration 2023-05-02 02:54:10,199 DEBUG [RS:0;jenkins-hbase4:39081-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 7d9d593eeacf44998515b23aa082bc6e: NONE, but ROW specified in column family configuration 2023-05-02 02:54:10,199 INFO [RS:0;jenkins-hbase4:39081-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 024f19513d3a9ded6c060054f1607e68/cf2 of 024f19513d3a9ded6c060054f1607e68 into 326e4bf0cbf6466c986237da67de478c(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-02 02:54:10,199 DEBUG [RS:0;jenkins-hbase4:39081-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 024f19513d3a9ded6c060054f1607e68: 2023-05-02 02:54:10,199 INFO [RS:0;jenkins-hbase4:39081-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1682996048869.024f19513d3a9ded6c060054f1607e68., storeName=024f19513d3a9ded6c060054f1607e68/cf2, priority=9, startTime=1682996050138; duration=0sec 2023-05-02 02:54:10,200 DEBUG [RS:0;jenkins-hbase4:39081-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-02 02:54:10,200 INFO [RS:0;jenkins-hbase4:39081-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 024f19513d3a9ded6c060054f1607e68/cf1 of 024f19513d3a9ded6c060054f1607e68 into 7d9d593eeacf44998515b23aa082bc6e(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-02 02:54:10,200 DEBUG [RS:0;jenkins-hbase4:39081-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 024f19513d3a9ded6c060054f1607e68: 2023-05-02 02:54:10,200 INFO [RS:0;jenkins-hbase4:39081-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1682996048869.024f19513d3a9ded6c060054f1607e68., storeName=024f19513d3a9ded6c060054f1607e68/cf1, priority=9, startTime=1682996050137; duration=0sec 2023-05-02 02:54:10,200 DEBUG [RS:0;jenkins-hbase4:39081-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-02 02:54:10,207 DEBUG [StoreOpener-bb895e1df53cda46bfcb5f94fe1710c9-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf2/76b4e49972d544bdb7a8702299d57cc6.8af7d4111b21c70efea08f7beab73a9f->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/76b4e49972d544bdb7a8702299d57cc6-bottom 2023-05-02 02:54:10,211 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a02bf38bab7c419c8ba5d1fb45d02f4c.8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:10,211 DEBUG [StoreOpener-bb895e1df53cda46bfcb5f94fe1710c9-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf2/a02bf38bab7c419c8ba5d1fb45d02f4c.8af7d4111b21c70efea08f7beab73a9f->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/a02bf38bab7c419c8ba5d1fb45d02f4c-bottom 2023-05-02 02:54:10,214 DEBUG [StoreOpener-bb895e1df53cda46bfcb5f94fe1710c9-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf2/e6bbd81d56884f75a2ab34f638d3dc7f.8af7d4111b21c70efea08f7beab73a9f->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/e6bbd81d56884f75a2ab34f638d3dc7f-bottom 2023-05-02 02:54:10,217 DEBUG [StoreOpener-bb895e1df53cda46bfcb5f94fe1710c9-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf2/eecb4f6bc8094e669b2ee7981bc5a39f.8af7d4111b21c70efea08f7beab73a9f->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/eecb4f6bc8094e669b2ee7981bc5a39f-bottom 2023-05-02 02:54:10,220 DEBUG [StoreOpener-bb895e1df53cda46bfcb5f94fe1710c9-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf2/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-378b3c3bb4df4380b5f3244397d9fcb2 2023-05-02 02:54:10,223 DEBUG [StoreOpener-bb895e1df53cda46bfcb5f94fe1710c9-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf2/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-8243f44fc750458da547c0947dff42a0 2023-05-02 02:54:10,226 DEBUG [StoreOpener-bb895e1df53cda46bfcb5f94fe1710c9-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf2/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-f894abdeb02b4e87bc8789e67b483cc0 2023-05-02 02:54:10,226 INFO [StoreOpener-bb895e1df53cda46bfcb5f94fe1710c9-1] regionserver.HStore(310): Store=bb895e1df53cda46bfcb5f94fe1710c9/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-02 02:54:10,227 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9 2023-05-02 02:54:10,228 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9 2023-05-02 02:54:10,230 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-02 02:54:10,231 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for bb895e1df53cda46bfcb5f94fe1710c9 2023-05-02 02:54:10,232 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened bb895e1df53cda46bfcb5f94fe1710c9; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=58832258, jitterRate=-0.12333104014396667}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-02 02:54:10,232 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for bb895e1df53cda46bfcb5f94fe1710c9: 2023-05-02 02:54:10,232 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1682996048869.bb895e1df53cda46bfcb5f94fe1710c9., pid=77, masterSystemTime=1682996050049 2023-05-02 02:54:10,233 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-02 02:54:10,234 DEBUG [RS:0;jenkins-hbase4:39081-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-02 02:54:10,234 DEBUG [RS:0;jenkins-hbase4:39081-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-02 02:54:10,234 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-05-02 02:54:10,234 INFO [RS:0;jenkins-hbase4:39081-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1682996048869.bb895e1df53cda46bfcb5f94fe1710c9. 2023-05-02 02:54:10,234 INFO [RS:0;jenkins-hbase4:39081-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1682996048869.bb895e1df53cda46bfcb5f94fe1710c9. 2023-05-02 02:54:10,234 DEBUG [RS:0;jenkins-hbase4:39081-shortCompactions-0] regionserver.HStore(1912): bb895e1df53cda46bfcb5f94fe1710c9/cf1 is initiating minor compaction (all files) 2023-05-02 02:54:10,234 DEBUG [RS:0;jenkins-hbase4:39081-longCompactions-0] regionserver.HStore(1912): bb895e1df53cda46bfcb5f94fe1710c9/cf2 is initiating minor compaction (all files) 2023-05-02 02:54:10,234 INFO [RS:0;jenkins-hbase4:39081-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of bb895e1df53cda46bfcb5f94fe1710c9/cf1 in testSplitTableRegionDeletedRowsDaughter,,1682996048869.bb895e1df53cda46bfcb5f94fe1710c9. 2023-05-02 02:54:10,234 INFO [RS:0;jenkins-hbase4:39081-longCompactions-0] regionserver.HRegion(2259): Starting compaction of bb895e1df53cda46bfcb5f94fe1710c9/cf2 in testSplitTableRegionDeletedRowsDaughter,,1682996048869.bb895e1df53cda46bfcb5f94fe1710c9. 2023-05-02 02:54:10,234 INFO [RS:0;jenkins-hbase4:39081-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf1/3189d0758d394d4290511c7d95e297bb.8af7d4111b21c70efea08f7beab73a9f->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/3189d0758d394d4290511c7d95e297bb-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf1/f6fbf066cf0a4af6b2ea76697cc904da.8af7d4111b21c70efea08f7beab73a9f->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/f6fbf066cf0a4af6b2ea76697cc904da-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf1/fa00f661973547868b8b686731148b94.8af7d4111b21c70efea08f7beab73a9f->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/fa00f661973547868b8b686731148b94-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf1/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-b0c982e0384d423aa455befed4ca21c3, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf1/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-609652eb416344f8b705e1e96171ce1e, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf1/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-40f1a5f529284647a6cf211a61001756, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf1/8db73eea04c64e878c8c4529b89946df.8af7d4111b21c70efea08f7beab73a9f->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/8db73eea04c64e878c8c4529b89946df-bottom] into tmpdir=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/.tmp, totalSize=35.0 K 2023-05-02 02:54:10,234 INFO [RS:0;jenkins-hbase4:39081-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf2/76b4e49972d544bdb7a8702299d57cc6.8af7d4111b21c70efea08f7beab73a9f->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/76b4e49972d544bdb7a8702299d57cc6-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf2/e6bbd81d56884f75a2ab34f638d3dc7f.8af7d4111b21c70efea08f7beab73a9f->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/e6bbd81d56884f75a2ab34f638d3dc7f-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf2/eecb4f6bc8094e669b2ee7981bc5a39f.8af7d4111b21c70efea08f7beab73a9f->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/eecb4f6bc8094e669b2ee7981bc5a39f-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf2/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-f894abdeb02b4e87bc8789e67b483cc0, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf2/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-8243f44fc750458da547c0947dff42a0, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf2/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-378b3c3bb4df4380b5f3244397d9fcb2, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf2/a02bf38bab7c419c8ba5d1fb45d02f4c.8af7d4111b21c70efea08f7beab73a9f->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/a02bf38bab7c419c8ba5d1fb45d02f4c-bottom] into tmpdir=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/.tmp, totalSize=35.0 K 2023-05-02 02:54:10,235 DEBUG [RS:0;jenkins-hbase4:39081-shortCompactions-0] compactions.Compactor(207): Compacting 3189d0758d394d4290511c7d95e297bb.8af7d4111b21c70efea08f7beab73a9f, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1682996047185 2023-05-02 02:54:10,235 DEBUG [RS:0;jenkins-hbase4:39081-longCompactions-0] compactions.Compactor(207): Compacting 76b4e49972d544bdb7a8702299d57cc6.8af7d4111b21c70efea08f7beab73a9f, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1682996047185 2023-05-02 02:54:10,235 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1682996048869.bb895e1df53cda46bfcb5f94fe1710c9. 2023-05-02 02:54:10,235 DEBUG [RS:0;jenkins-hbase4:39081-shortCompactions-0] compactions.Compactor(207): Compacting f6fbf066cf0a4af6b2ea76697cc904da.8af7d4111b21c70efea08f7beab73a9f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1682996047401 2023-05-02 02:54:10,235 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionDeletedRowsDaughter,,1682996048869.bb895e1df53cda46bfcb5f94fe1710c9. 2023-05-02 02:54:10,236 DEBUG [RS:0;jenkins-hbase4:39081-longCompactions-0] compactions.Compactor(207): Compacting e6bbd81d56884f75a2ab34f638d3dc7f.8af7d4111b21c70efea08f7beab73a9f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1682996047401 2023-05-02 02:54:10,236 DEBUG [RS:0;jenkins-hbase4:39081-shortCompactions-0] compactions.Compactor(207): Compacting fa00f661973547868b8b686731148b94.8af7d4111b21c70efea08f7beab73a9f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1682996047578 2023-05-02 02:54:10,236 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=bb895e1df53cda46bfcb5f94fe1710c9, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:10,236 DEBUG [RS:0;jenkins-hbase4:39081-longCompactions-0] compactions.Compactor(207): Compacting eecb4f6bc8094e669b2ee7981bc5a39f.8af7d4111b21c70efea08f7beab73a9f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1682996047578 2023-05-02 02:54:10,236 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1682996048869.bb895e1df53cda46bfcb5f94fe1710c9.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1682996050236"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1682996050236"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1682996050236"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682996050236"}]},"ts":"1682996050236"} 2023-05-02 02:54:10,236 DEBUG [RS:0;jenkins-hbase4:39081-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-b0c982e0384d423aa455befed4ca21c3, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1682996047759 2023-05-02 02:54:10,236 DEBUG [RS:0;jenkins-hbase4:39081-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-f894abdeb02b4e87bc8789e67b483cc0, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1682996047759 2023-05-02 02:54:10,237 DEBUG [RS:0;jenkins-hbase4:39081-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-609652eb416344f8b705e1e96171ce1e, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1682996047945 2023-05-02 02:54:10,237 DEBUG [RS:0;jenkins-hbase4:39081-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-40f1a5f529284647a6cf211a61001756, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1682996048127 2023-05-02 02:54:10,237 DEBUG [RS:0;jenkins-hbase4:39081-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-8243f44fc750458da547c0947dff42a0, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1682996047945 2023-05-02 02:54:10,237 DEBUG [RS:0;jenkins-hbase4:39081-shortCompactions-0] compactions.Compactor(207): Compacting 8db73eea04c64e878c8c4529b89946df.8af7d4111b21c70efea08f7beab73a9f, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1682996048321 2023-05-02 02:54:10,237 DEBUG [RS:0;jenkins-hbase4:39081-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-378b3c3bb4df4380b5f3244397d9fcb2, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1682996048127 2023-05-02 02:54:10,238 DEBUG [RS:0;jenkins-hbase4:39081-longCompactions-0] compactions.Compactor(207): Compacting a02bf38bab7c419c8ba5d1fb45d02f4c.8af7d4111b21c70efea08f7beab73a9f, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1682996048321 2023-05-02 02:54:10,240 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=77, resume processing ppid=75 2023-05-02 02:54:10,240 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=77, ppid=75, state=SUCCESS; OpenRegionProcedure bb895e1df53cda46bfcb5f94fe1710c9, server=jenkins-hbase4.apache.org,39081,1682996021579 in 340 msec 2023-05-02 02:54:10,242 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=75, resume processing ppid=72 2023-05-02 02:54:10,242 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=75, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=bb895e1df53cda46bfcb5f94fe1710c9, ASSIGN in 498 msec 2023-05-02 02:54:10,244 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=8af7d4111b21c70efea08f7beab73a9f, daughterA=bb895e1df53cda46bfcb5f94fe1710c9, daughterB=024f19513d3a9ded6c060054f1607e68 in 1.3730 sec 2023-05-02 02:54:10,254 INFO [RS:0;jenkins-hbase4:39081-longCompactions-0] throttle.PressureAwareThroughputController(145): bb895e1df53cda46bfcb5f94fe1710c9#cf2#compaction#74 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-02 02:54:10,256 INFO [RS:0;jenkins-hbase4:39081-shortCompactions-0] throttle.PressureAwareThroughputController(145): bb895e1df53cda46bfcb5f94fe1710c9#cf1#compaction#75 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-02 02:54:10,274 DEBUG [RS:0;jenkins-hbase4:39081-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/.tmp/cf2/a28114ecfca7457f9ef9137dbf838131 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf2/a28114ecfca7457f9ef9137dbf838131 2023-05-02 02:54:10,275 DEBUG [RS:0;jenkins-hbase4:39081-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/.tmp/cf1/fb36b3979da84e00b0b51a9d0913cf7a as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf1/fb36b3979da84e00b0b51a9d0913cf7a 2023-05-02 02:54:10,279 INFO [RS:0;jenkins-hbase4:39081-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in bb895e1df53cda46bfcb5f94fe1710c9/cf2 of bb895e1df53cda46bfcb5f94fe1710c9 into a28114ecfca7457f9ef9137dbf838131(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-02 02:54:10,279 DEBUG [RS:0;jenkins-hbase4:39081-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for bb895e1df53cda46bfcb5f94fe1710c9: 2023-05-02 02:54:10,279 INFO [RS:0;jenkins-hbase4:39081-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1682996048869.bb895e1df53cda46bfcb5f94fe1710c9., storeName=bb895e1df53cda46bfcb5f94fe1710c9/cf2, priority=9, startTime=1682996050234; duration=0sec 2023-05-02 02:54:10,280 DEBUG [RS:0;jenkins-hbase4:39081-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-02 02:54:10,280 INFO [RS:0;jenkins-hbase4:39081-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in bb895e1df53cda46bfcb5f94fe1710c9/cf1 of bb895e1df53cda46bfcb5f94fe1710c9 into fb36b3979da84e00b0b51a9d0913cf7a(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-02 02:54:10,280 DEBUG [RS:0;jenkins-hbase4:39081-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for bb895e1df53cda46bfcb5f94fe1710c9: 2023-05-02 02:54:10,280 INFO [RS:0;jenkins-hbase4:39081-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1682996048869.bb895e1df53cda46bfcb5f94fe1710c9., storeName=bb895e1df53cda46bfcb5f94fe1710c9/cf1, priority=9, startTime=1682996050233; duration=0sec 2023-05-02 02:54:10,280 DEBUG [RS:0;jenkins-hbase4:39081-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-02 02:54:10,384 DEBUG [Listener at localhost/37119] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-05-02 02:54:10,387 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60398, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-05-02 02:54:10,387 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39081] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,,1682996048869.bb895e1df53cda46bfcb5f94fe1710c9. 2023-05-02 02:54:10,388 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39081] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-05-02 02:54:10,388 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39081] regionserver.HStore(1912): bb895e1df53cda46bfcb5f94fe1710c9/cf1 is initiating major compaction (all files) 2023-05-02 02:54:10,389 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39081] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@6b2f1162; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1682996048869.bb895e1df53cda46bfcb5f94fe1710c9.; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-02 02:54:10,389 INFO [RS:0;jenkins-hbase4:39081-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of bb895e1df53cda46bfcb5f94fe1710c9/cf1 in testSplitTableRegionDeletedRowsDaughter,,1682996048869.bb895e1df53cda46bfcb5f94fe1710c9. 2023-05-02 02:54:10,389 INFO [RS:0;jenkins-hbase4:39081-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf1/fb36b3979da84e00b0b51a9d0913cf7a] into tmpdir=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/.tmp, totalSize=6.8 K 2023-05-02 02:54:10,389 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39081] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-05-02 02:54:10,389 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39081] regionserver.HStore(1912): bb895e1df53cda46bfcb5f94fe1710c9/cf2 is initiating major compaction (all files) 2023-05-02 02:54:10,389 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39081] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@6086d4f6; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1682996048869.bb895e1df53cda46bfcb5f94fe1710c9.; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-05-02 02:54:10,389 INFO [RS:0;jenkins-hbase4:39081-longCompactions-0] regionserver.HRegion(2259): Starting compaction of bb895e1df53cda46bfcb5f94fe1710c9/cf2 in testSplitTableRegionDeletedRowsDaughter,,1682996048869.bb895e1df53cda46bfcb5f94fe1710c9. 2023-05-02 02:54:10,389 INFO [RS:0;jenkins-hbase4:39081-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf2/a28114ecfca7457f9ef9137dbf838131] into tmpdir=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/.tmp, totalSize=6.8 K 2023-05-02 02:54:10,389 DEBUG [RS:0;jenkins-hbase4:39081-shortCompactions-0] compactions.Compactor(207): Compacting fb36b3979da84e00b0b51a9d0913cf7a, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1682996047185 2023-05-02 02:54:10,390 DEBUG [RS:0;jenkins-hbase4:39081-longCompactions-0] compactions.Compactor(207): Compacting a28114ecfca7457f9ef9137dbf838131, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1682996047185 2023-05-02 02:54:10,391 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39081] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,60,1682996048869.024f19513d3a9ded6c060054f1607e68. 2023-05-02 02:54:10,391 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39081] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-05-02 02:54:10,392 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39081] regionserver.HStore(1912): 024f19513d3a9ded6c060054f1607e68/cf1 is initiating major compaction (all files) 2023-05-02 02:54:10,392 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39081] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@40f35f02; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,60,1682996048869.024f19513d3a9ded6c060054f1607e68.; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-05-02 02:54:10,392 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39081] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-05-02 02:54:10,392 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39081] regionserver.HStore(1912): 024f19513d3a9ded6c060054f1607e68/cf2 is initiating major compaction (all files) 2023-05-02 02:54:10,392 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39081] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@7a82075b; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,60,1682996048869.024f19513d3a9ded6c060054f1607e68.; compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-05-02 02:54:10,393 INFO [Listener at localhost/37119] hbase.Waiter(180): Waiting up to [6,000] milli-secs(wait.for.ratio=[1]) 2023-05-02 02:54:10,395 INFO [RS:0;jenkins-hbase4:39081-longCompactions-0] throttle.PressureAwareThroughputController(145): bb895e1df53cda46bfcb5f94fe1710c9#cf2#compaction#76 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-05-02 02:54:10,395 INFO [RS:0;jenkins-hbase4:39081-shortCompactions-0] throttle.PressureAwareThroughputController(145): bb895e1df53cda46bfcb5f94fe1710c9#cf1#compaction#77 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-02 02:54:10,416 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39081] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,,1682996048869.bb895e1df53cda46bfcb5f94fe1710c9. is not mergeable because it has references 2023-05-02 02:54:10,417 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39081] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,60,1682996048869.024f19513d3a9ded6c060054f1607e68. is not mergeable because it has references 2023-05-02 02:54:10,421 DEBUG [RS:0;jenkins-hbase4:39081-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/.tmp/cf2/046985a41802469ba7607724dfd06d98 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf2/046985a41802469ba7607724dfd06d98 2023-05-02 02:54:10,422 DEBUG [RS:0;jenkins-hbase4:39081-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/.tmp/cf1/7c833b141ad64d17a54fa2a57618d1fc as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf1/7c833b141ad64d17a54fa2a57618d1fc 2023-05-02 02:54:10,426 INFO [RS:0;jenkins-hbase4:39081-longCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in bb895e1df53cda46bfcb5f94fe1710c9/cf2 of bb895e1df53cda46bfcb5f94fe1710c9 into 046985a41802469ba7607724dfd06d98(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-02 02:54:10,426 DEBUG [RS:0;jenkins-hbase4:39081-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for bb895e1df53cda46bfcb5f94fe1710c9: 2023-05-02 02:54:10,426 INFO [RS:0;jenkins-hbase4:39081-shortCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in bb895e1df53cda46bfcb5f94fe1710c9/cf1 of bb895e1df53cda46bfcb5f94fe1710c9 into 7c833b141ad64d17a54fa2a57618d1fc(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-02 02:54:10,427 DEBUG [RS:0;jenkins-hbase4:39081-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for bb895e1df53cda46bfcb5f94fe1710c9: 2023-05-02 02:54:10,427 INFO [RS:0;jenkins-hbase4:39081-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1682996048869.bb895e1df53cda46bfcb5f94fe1710c9., storeName=cf2, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1682996050389; duration=0sec 2023-05-02 02:54:10,427 INFO [RS:0;jenkins-hbase4:39081-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1682996048869.bb895e1df53cda46bfcb5f94fe1710c9., storeName=cf1, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1682996050388; duration=0sec 2023-05-02 02:54:10,427 DEBUG [RS:0;jenkins-hbase4:39081-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-05-02 02:54:10,427 DEBUG [RS:0;jenkins-hbase4:39081-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-05-02 02:54:10,427 INFO [RS:0;jenkins-hbase4:39081-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 024f19513d3a9ded6c060054f1607e68/cf2 in testSplitTableRegionDeletedRowsDaughter,60,1682996048869.024f19513d3a9ded6c060054f1607e68. 2023-05-02 02:54:10,427 INFO [RS:0;jenkins-hbase4:39081-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 024f19513d3a9ded6c060054f1607e68/cf1 in testSplitTableRegionDeletedRowsDaughter,60,1682996048869.024f19513d3a9ded6c060054f1607e68. 2023-05-02 02:54:10,427 INFO [RS:0;jenkins-hbase4:39081-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf2/326e4bf0cbf6466c986237da67de478c] into tmpdir=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/.tmp, totalSize=5.0 K 2023-05-02 02:54:10,427 INFO [RS:0;jenkins-hbase4:39081-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf1/7d9d593eeacf44998515b23aa082bc6e] into tmpdir=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/.tmp, totalSize=5.0 K 2023-05-02 02:54:10,428 DEBUG [RS:0;jenkins-hbase4:39081-longCompactions-0] compactions.Compactor(207): Compacting 326e4bf0cbf6466c986237da67de478c, keycount=0, bloomtype=NONE, size=5.0 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-05-02 02:54:10,428 DEBUG [RS:0;jenkins-hbase4:39081-shortCompactions-0] compactions.Compactor(207): Compacting 7d9d593eeacf44998515b23aa082bc6e, keycount=0, bloomtype=NONE, size=5.0 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-05-02 02:54:10,430 INFO [RS:0;jenkins-hbase4:39081-shortCompactions-0] throttle.PressureAwareThroughputController(145): 024f19513d3a9ded6c060054f1607e68#cf1#compaction#78 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-02 02:54:10,430 INFO [RS:0;jenkins-hbase4:39081-longCompactions-0] throttle.PressureAwareThroughputController(145): 024f19513d3a9ded6c060054f1607e68#cf2#compaction#79 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-02 02:54:10,444 DEBUG [RS:0;jenkins-hbase4:39081-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 5a14c4b25f9c46ed9fea0c2b36698e80: NONE, but ROW specified in column family configuration 2023-05-02 02:54:10,445 DEBUG [RS:0;jenkins-hbase4:39081-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/.tmp/cf1/5a14c4b25f9c46ed9fea0c2b36698e80 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf1/5a14c4b25f9c46ed9fea0c2b36698e80 2023-05-02 02:54:10,449 DEBUG [RS:0;jenkins-hbase4:39081-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 847c4a3c095d47c692df905147536de6: NONE, but ROW specified in column family configuration 2023-05-02 02:54:10,449 DEBUG [RS:0;jenkins-hbase4:39081-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 5a14c4b25f9c46ed9fea0c2b36698e80: NONE, but ROW specified in column family configuration 2023-05-02 02:54:10,450 DEBUG [RS:0;jenkins-hbase4:39081-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/.tmp/cf2/847c4a3c095d47c692df905147536de6 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf2/847c4a3c095d47c692df905147536de6 2023-05-02 02:54:10,450 INFO [RS:0;jenkins-hbase4:39081-shortCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 024f19513d3a9ded6c060054f1607e68/cf1 of 024f19513d3a9ded6c060054f1607e68 into 5a14c4b25f9c46ed9fea0c2b36698e80(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-02 02:54:10,450 DEBUG [RS:0;jenkins-hbase4:39081-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 024f19513d3a9ded6c060054f1607e68: 2023-05-02 02:54:10,450 INFO [RS:0;jenkins-hbase4:39081-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,60,1682996048869.024f19513d3a9ded6c060054f1607e68., storeName=cf1, fileCount=1, fileSize=5.0 K (5.0 K), priority=1, time=1682996050392; duration=0sec 2023-05-02 02:54:10,450 DEBUG [RS:0;jenkins-hbase4:39081-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-02 02:54:10,454 DEBUG [RS:0;jenkins-hbase4:39081-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 847c4a3c095d47c692df905147536de6: NONE, but ROW specified in column family configuration 2023-05-02 02:54:10,455 INFO [RS:0;jenkins-hbase4:39081-longCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 024f19513d3a9ded6c060054f1607e68/cf2 of 024f19513d3a9ded6c060054f1607e68 into 847c4a3c095d47c692df905147536de6(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-02 02:54:10,455 DEBUG [RS:0;jenkins-hbase4:39081-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 024f19513d3a9ded6c060054f1607e68: 2023-05-02 02:54:10,455 INFO [RS:0;jenkins-hbase4:39081-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,60,1682996048869.024f19513d3a9ded6c060054f1607e68., storeName=cf2, fileCount=1, fileSize=5.0 K (5.0 K), priority=1, time=1682996050392; duration=0sec 2023-05-02 02:54:10,455 DEBUG [RS:0;jenkins-hbase4:39081-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-02 02:54:10,526 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39081] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,,1682996048869.bb895e1df53cda46bfcb5f94fe1710c9. is not mergeable because it has references 2023-05-02 02:54:10,526 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39081] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,60,1682996048869.024f19513d3a9ded6c060054f1607e68. is not mergeable because it has references 2023-05-02 02:54:10,532 DEBUG [Listener at localhost/37119] client.ClientScanner(240): Advancing internal scanner to startKey at '60', inclusive 2023-05-02 02:54:10,534 WARN [Listener at localhost/37119] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-02 02:54:10,535 INFO [Listener at localhost/37119] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:10,536 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:10,537 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure2.ProcedureExecutor(1029): Stored pid=79, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:10,539 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-05-02 02:54:10,539 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682996050539"}]},"ts":"1682996050539"} 2023-05-02 02:54:10,540 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLING in hbase:meta 2023-05-02 02:54:10,542 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLING 2023-05-02 02:54:10,542 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=bb895e1df53cda46bfcb5f94fe1710c9, UNASSIGN}, {pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=024f19513d3a9ded6c060054f1607e68, UNASSIGN}] 2023-05-02 02:54:10,543 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=024f19513d3a9ded6c060054f1607e68, UNASSIGN 2023-05-02 02:54:10,544 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=024f19513d3a9ded6c060054f1607e68, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:10,544 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1682996048869.024f19513d3a9ded6c060054f1607e68.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1682996050544"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1682996050544"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682996050544"}]},"ts":"1682996050544"} 2023-05-02 02:54:10,545 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=82, ppid=81, state=RUNNABLE; CloseRegionProcedure 024f19513d3a9ded6c060054f1607e68, server=jenkins-hbase4.apache.org,39081,1682996021579}] 2023-05-02 02:54:10,547 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=bb895e1df53cda46bfcb5f94fe1710c9, UNASSIGN 2023-05-02 02:54:10,547 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=bb895e1df53cda46bfcb5f94fe1710c9, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:10,547 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1682996048869.bb895e1df53cda46bfcb5f94fe1710c9.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1682996050547"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1682996050547"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682996050547"}]},"ts":"1682996050547"} 2023-05-02 02:54:10,548 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=83, ppid=80, state=RUNNABLE; CloseRegionProcedure bb895e1df53cda46bfcb5f94fe1710c9, server=jenkins-hbase4.apache.org,39081,1682996021579}] 2023-05-02 02:54:10,640 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-05-02 02:54:10,697 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 024f19513d3a9ded6c060054f1607e68 2023-05-02 02:54:10,697 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 024f19513d3a9ded6c060054f1607e68, disabling compactions & flushes 2023-05-02 02:54:10,697 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionDeletedRowsDaughter,60,1682996048869.024f19513d3a9ded6c060054f1607e68. 2023-05-02 02:54:10,697 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,60,1682996048869.024f19513d3a9ded6c060054f1607e68. 2023-05-02 02:54:10,697 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,60,1682996048869.024f19513d3a9ded6c060054f1607e68. after waiting 0 ms 2023-05-02 02:54:10,697 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,60,1682996048869.024f19513d3a9ded6c060054f1607e68. 2023-05-02 02:54:10,698 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1682996048869.024f19513d3a9ded6c060054f1607e68.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf1/3189d0758d394d4290511c7d95e297bb.8af7d4111b21c70efea08f7beab73a9f->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/3189d0758d394d4290511c7d95e297bb-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf1/f6fbf066cf0a4af6b2ea76697cc904da.8af7d4111b21c70efea08f7beab73a9f->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/f6fbf066cf0a4af6b2ea76697cc904da-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf1/fa00f661973547868b8b686731148b94.8af7d4111b21c70efea08f7beab73a9f->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/fa00f661973547868b8b686731148b94-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf1/8db73eea04c64e878c8c4529b89946df.8af7d4111b21c70efea08f7beab73a9f->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/8db73eea04c64e878c8c4529b89946df-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf1/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-d75d441bf5874fe6b71adefb02039137, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf1/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-72ce99472ae84721bfaf8e780648de42, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf1/7d9d593eeacf44998515b23aa082bc6e, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf1/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-dc659b190e7744abb3e0fe388448fa31] to archive 2023-05-02 02:54:10,698 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1682996048869.024f19513d3a9ded6c060054f1607e68.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-02 02:54:10,700 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1682996048869.024f19513d3a9ded6c060054f1607e68.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf1/3189d0758d394d4290511c7d95e297bb.8af7d4111b21c70efea08f7beab73a9f to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf1/3189d0758d394d4290511c7d95e297bb.8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:10,701 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1682996048869.024f19513d3a9ded6c060054f1607e68.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf1/f6fbf066cf0a4af6b2ea76697cc904da.8af7d4111b21c70efea08f7beab73a9f to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf1/f6fbf066cf0a4af6b2ea76697cc904da.8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:10,702 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1682996048869.024f19513d3a9ded6c060054f1607e68.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf1/fa00f661973547868b8b686731148b94.8af7d4111b21c70efea08f7beab73a9f to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf1/fa00f661973547868b8b686731148b94.8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:10,703 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1682996048869.024f19513d3a9ded6c060054f1607e68.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf1/8db73eea04c64e878c8c4529b89946df.8af7d4111b21c70efea08f7beab73a9f to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf1/8db73eea04c64e878c8c4529b89946df.8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:10,704 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1682996048869.024f19513d3a9ded6c060054f1607e68.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf1/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-d75d441bf5874fe6b71adefb02039137 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf1/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-d75d441bf5874fe6b71adefb02039137 2023-05-02 02:54:10,705 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1682996048869.024f19513d3a9ded6c060054f1607e68.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf1/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-72ce99472ae84721bfaf8e780648de42 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf1/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-72ce99472ae84721bfaf8e780648de42 2023-05-02 02:54:10,705 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1682996048869.024f19513d3a9ded6c060054f1607e68.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf1/7d9d593eeacf44998515b23aa082bc6e to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf1/7d9d593eeacf44998515b23aa082bc6e 2023-05-02 02:54:10,706 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1682996048869.024f19513d3a9ded6c060054f1607e68.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf1/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-dc659b190e7744abb3e0fe388448fa31 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf1/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-dc659b190e7744abb3e0fe388448fa31 2023-05-02 02:54:10,708 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1682996048869.024f19513d3a9ded6c060054f1607e68.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf2/76b4e49972d544bdb7a8702299d57cc6.8af7d4111b21c70efea08f7beab73a9f->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/76b4e49972d544bdb7a8702299d57cc6-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf2/e6bbd81d56884f75a2ab34f638d3dc7f.8af7d4111b21c70efea08f7beab73a9f->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/e6bbd81d56884f75a2ab34f638d3dc7f-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf2/eecb4f6bc8094e669b2ee7981bc5a39f.8af7d4111b21c70efea08f7beab73a9f->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/eecb4f6bc8094e669b2ee7981bc5a39f-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf2/a02bf38bab7c419c8ba5d1fb45d02f4c.8af7d4111b21c70efea08f7beab73a9f->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/a02bf38bab7c419c8ba5d1fb45d02f4c-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf2/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-eb4bc94f1cf94bc080443cc926ed2a17, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf2/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-09bd463709ed4f1e83a4a0d35349012e, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf2/326e4bf0cbf6466c986237da67de478c, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf2/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-90a8fcb9bc36402ab2af45c21a1f4254] to archive 2023-05-02 02:54:10,709 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1682996048869.024f19513d3a9ded6c060054f1607e68.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-02 02:54:10,710 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1682996048869.024f19513d3a9ded6c060054f1607e68.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf2/76b4e49972d544bdb7a8702299d57cc6.8af7d4111b21c70efea08f7beab73a9f to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf2/76b4e49972d544bdb7a8702299d57cc6.8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:10,711 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1682996048869.024f19513d3a9ded6c060054f1607e68.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf2/e6bbd81d56884f75a2ab34f638d3dc7f.8af7d4111b21c70efea08f7beab73a9f to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf2/e6bbd81d56884f75a2ab34f638d3dc7f.8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:10,711 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1682996048869.024f19513d3a9ded6c060054f1607e68.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf2/eecb4f6bc8094e669b2ee7981bc5a39f.8af7d4111b21c70efea08f7beab73a9f to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf2/eecb4f6bc8094e669b2ee7981bc5a39f.8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:10,713 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1682996048869.024f19513d3a9ded6c060054f1607e68.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf2/a02bf38bab7c419c8ba5d1fb45d02f4c.8af7d4111b21c70efea08f7beab73a9f to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf2/a02bf38bab7c419c8ba5d1fb45d02f4c.8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:10,714 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1682996048869.024f19513d3a9ded6c060054f1607e68.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf2/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-eb4bc94f1cf94bc080443cc926ed2a17 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf2/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-eb4bc94f1cf94bc080443cc926ed2a17 2023-05-02 02:54:10,715 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1682996048869.024f19513d3a9ded6c060054f1607e68.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf2/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-09bd463709ed4f1e83a4a0d35349012e to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf2/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-09bd463709ed4f1e83a4a0d35349012e 2023-05-02 02:54:10,716 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1682996048869.024f19513d3a9ded6c060054f1607e68.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf2/326e4bf0cbf6466c986237da67de478c to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf2/326e4bf0cbf6466c986237da67de478c 2023-05-02 02:54:10,717 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1682996048869.024f19513d3a9ded6c060054f1607e68.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf2/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-90a8fcb9bc36402ab2af45c21a1f4254 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf2/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-90a8fcb9bc36402ab2af45c21a1f4254 2023-05-02 02:54:10,720 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=106 2023-05-02 02:54:10,721 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-02 02:54:10,721 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionDeletedRowsDaughter,60,1682996048869.024f19513d3a9ded6c060054f1607e68. 2023-05-02 02:54:10,721 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 024f19513d3a9ded6c060054f1607e68: 2023-05-02 02:54:10,722 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 024f19513d3a9ded6c060054f1607e68 2023-05-02 02:54:10,722 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close bb895e1df53cda46bfcb5f94fe1710c9 2023-05-02 02:54:10,722 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing bb895e1df53cda46bfcb5f94fe1710c9, disabling compactions & flushes 2023-05-02 02:54:10,722 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionDeletedRowsDaughter,,1682996048869.bb895e1df53cda46bfcb5f94fe1710c9. 2023-05-02 02:54:10,722 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1682996048869.bb895e1df53cda46bfcb5f94fe1710c9. 2023-05-02 02:54:10,722 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1682996048869.bb895e1df53cda46bfcb5f94fe1710c9. after waiting 0 ms 2023-05-02 02:54:10,722 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1682996048869.bb895e1df53cda46bfcb5f94fe1710c9. 2023-05-02 02:54:10,723 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=024f19513d3a9ded6c060054f1607e68, regionState=CLOSED 2023-05-02 02:54:10,723 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,60,1682996048869.024f19513d3a9ded6c060054f1607e68.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1682996050723"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682996050723"}]},"ts":"1682996050723"} 2023-05-02 02:54:10,723 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1682996048869.bb895e1df53cda46bfcb5f94fe1710c9.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf1/3189d0758d394d4290511c7d95e297bb.8af7d4111b21c70efea08f7beab73a9f->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/3189d0758d394d4290511c7d95e297bb-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf1/f6fbf066cf0a4af6b2ea76697cc904da.8af7d4111b21c70efea08f7beab73a9f->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/f6fbf066cf0a4af6b2ea76697cc904da-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf1/fa00f661973547868b8b686731148b94.8af7d4111b21c70efea08f7beab73a9f->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/fa00f661973547868b8b686731148b94-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf1/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-b0c982e0384d423aa455befed4ca21c3, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf1/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-609652eb416344f8b705e1e96171ce1e, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf1/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-40f1a5f529284647a6cf211a61001756, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf1/fb36b3979da84e00b0b51a9d0913cf7a, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf1/8db73eea04c64e878c8c4529b89946df.8af7d4111b21c70efea08f7beab73a9f->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/8db73eea04c64e878c8c4529b89946df-bottom] to archive 2023-05-02 02:54:10,724 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1682996048869.bb895e1df53cda46bfcb5f94fe1710c9.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-02 02:54:10,726 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1682996048869.bb895e1df53cda46bfcb5f94fe1710c9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf1/3189d0758d394d4290511c7d95e297bb.8af7d4111b21c70efea08f7beab73a9f to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf1/3189d0758d394d4290511c7d95e297bb.8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:10,726 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=82, resume processing ppid=81 2023-05-02 02:54:10,726 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=82, ppid=81, state=SUCCESS; CloseRegionProcedure 024f19513d3a9ded6c060054f1607e68, server=jenkins-hbase4.apache.org,39081,1682996021579 in 179 msec 2023-05-02 02:54:10,727 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=81, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=024f19513d3a9ded6c060054f1607e68, UNASSIGN in 184 msec 2023-05-02 02:54:10,727 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1682996048869.bb895e1df53cda46bfcb5f94fe1710c9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf1/f6fbf066cf0a4af6b2ea76697cc904da.8af7d4111b21c70efea08f7beab73a9f to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf1/f6fbf066cf0a4af6b2ea76697cc904da.8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:10,728 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1682996048869.bb895e1df53cda46bfcb5f94fe1710c9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf1/fa00f661973547868b8b686731148b94.8af7d4111b21c70efea08f7beab73a9f to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf1/fa00f661973547868b8b686731148b94.8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:10,729 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1682996048869.bb895e1df53cda46bfcb5f94fe1710c9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf1/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-b0c982e0384d423aa455befed4ca21c3 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf1/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-b0c982e0384d423aa455befed4ca21c3 2023-05-02 02:54:10,730 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1682996048869.bb895e1df53cda46bfcb5f94fe1710c9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf1/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-609652eb416344f8b705e1e96171ce1e to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf1/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-609652eb416344f8b705e1e96171ce1e 2023-05-02 02:54:10,731 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1682996048869.bb895e1df53cda46bfcb5f94fe1710c9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf1/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-40f1a5f529284647a6cf211a61001756 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf1/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-40f1a5f529284647a6cf211a61001756 2023-05-02 02:54:10,732 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1682996048869.bb895e1df53cda46bfcb5f94fe1710c9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf1/fb36b3979da84e00b0b51a9d0913cf7a to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf1/fb36b3979da84e00b0b51a9d0913cf7a 2023-05-02 02:54:10,733 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1682996048869.bb895e1df53cda46bfcb5f94fe1710c9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf1/8db73eea04c64e878c8c4529b89946df.8af7d4111b21c70efea08f7beab73a9f to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf1/8db73eea04c64e878c8c4529b89946df.8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:10,735 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1682996048869.bb895e1df53cda46bfcb5f94fe1710c9.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf2/76b4e49972d544bdb7a8702299d57cc6.8af7d4111b21c70efea08f7beab73a9f->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/76b4e49972d544bdb7a8702299d57cc6-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf2/e6bbd81d56884f75a2ab34f638d3dc7f.8af7d4111b21c70efea08f7beab73a9f->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/e6bbd81d56884f75a2ab34f638d3dc7f-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf2/eecb4f6bc8094e669b2ee7981bc5a39f.8af7d4111b21c70efea08f7beab73a9f->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/eecb4f6bc8094e669b2ee7981bc5a39f-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf2/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-f894abdeb02b4e87bc8789e67b483cc0, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf2/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-8243f44fc750458da547c0947dff42a0, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf2/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-378b3c3bb4df4380b5f3244397d9fcb2, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf2/a28114ecfca7457f9ef9137dbf838131, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf2/a02bf38bab7c419c8ba5d1fb45d02f4c.8af7d4111b21c70efea08f7beab73a9f->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/a02bf38bab7c419c8ba5d1fb45d02f4c-bottom] to archive 2023-05-02 02:54:10,736 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1682996048869.bb895e1df53cda46bfcb5f94fe1710c9.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-02 02:54:10,737 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1682996048869.bb895e1df53cda46bfcb5f94fe1710c9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf2/76b4e49972d544bdb7a8702299d57cc6.8af7d4111b21c70efea08f7beab73a9f to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf2/76b4e49972d544bdb7a8702299d57cc6.8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:10,738 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1682996048869.bb895e1df53cda46bfcb5f94fe1710c9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf2/e6bbd81d56884f75a2ab34f638d3dc7f.8af7d4111b21c70efea08f7beab73a9f to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf2/e6bbd81d56884f75a2ab34f638d3dc7f.8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:10,739 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1682996048869.bb895e1df53cda46bfcb5f94fe1710c9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf2/eecb4f6bc8094e669b2ee7981bc5a39f.8af7d4111b21c70efea08f7beab73a9f to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf2/eecb4f6bc8094e669b2ee7981bc5a39f.8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:10,740 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1682996048869.bb895e1df53cda46bfcb5f94fe1710c9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf2/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-f894abdeb02b4e87bc8789e67b483cc0 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf2/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-f894abdeb02b4e87bc8789e67b483cc0 2023-05-02 02:54:10,741 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1682996048869.bb895e1df53cda46bfcb5f94fe1710c9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf2/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-8243f44fc750458da547c0947dff42a0 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf2/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-8243f44fc750458da547c0947dff42a0 2023-05-02 02:54:10,741 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1682996048869.bb895e1df53cda46bfcb5f94fe1710c9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf2/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-378b3c3bb4df4380b5f3244397d9fcb2 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf2/testSplitTableRegionDeletedRowsDaughter=8af7d4111b21c70efea08f7beab73a9f-378b3c3bb4df4380b5f3244397d9fcb2 2023-05-02 02:54:10,742 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1682996048869.bb895e1df53cda46bfcb5f94fe1710c9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf2/a28114ecfca7457f9ef9137dbf838131 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf2/a28114ecfca7457f9ef9137dbf838131 2023-05-02 02:54:10,743 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1682996048869.bb895e1df53cda46bfcb5f94fe1710c9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf2/a02bf38bab7c419c8ba5d1fb45d02f4c.8af7d4111b21c70efea08f7beab73a9f to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf2/a02bf38bab7c419c8ba5d1fb45d02f4c.8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:10,746 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=106 2023-05-02 02:54:10,747 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-02 02:54:10,747 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionDeletedRowsDaughter,,1682996048869.bb895e1df53cda46bfcb5f94fe1710c9. 2023-05-02 02:54:10,747 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for bb895e1df53cda46bfcb5f94fe1710c9: 2023-05-02 02:54:10,748 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed bb895e1df53cda46bfcb5f94fe1710c9 2023-05-02 02:54:10,749 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=bb895e1df53cda46bfcb5f94fe1710c9, regionState=CLOSED 2023-05-02 02:54:10,749 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1682996048869.bb895e1df53cda46bfcb5f94fe1710c9.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1682996050749"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682996050749"}]},"ts":"1682996050749"} 2023-05-02 02:54:10,751 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=83, resume processing ppid=80 2023-05-02 02:54:10,752 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=83, ppid=80, state=SUCCESS; CloseRegionProcedure bb895e1df53cda46bfcb5f94fe1710c9, server=jenkins-hbase4.apache.org,39081,1682996021579 in 202 msec 2023-05-02 02:54:10,754 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=80, resume processing ppid=79 2023-05-02 02:54:10,754 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=80, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=bb895e1df53cda46bfcb5f94fe1710c9, UNASSIGN in 210 msec 2023-05-02 02:54:10,754 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682996050754"}]},"ts":"1682996050754"} 2023-05-02 02:54:10,755 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLED in hbase:meta 2023-05-02 02:54:10,757 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLED 2023-05-02 02:54:10,758 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 221 msec 2023-05-02 02:54:10,841 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-05-02 02:54:10,842 INFO [Listener at localhost/37119] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 79 completed 2023-05-02 02:54:10,842 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:10,843 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure2.ProcedureExecutor(1029): Stored pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:10,845 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:10,845 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=84, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:10,845 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1227): Checking to see if procedure is done pid=84 2023-05-02 02:54:10,850 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:10,850 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68 2023-05-02 02:54:10,850 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9 2023-05-02 02:54:10,852 DEBUG [HFileArchiver-2] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf1, FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf2, FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/recovered.edits] 2023-05-02 02:54:10,852 DEBUG [HFileArchiver-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf1, FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf2, FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/recovered.edits] 2023-05-02 02:54:10,855 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf1/7c833b141ad64d17a54fa2a57618d1fc to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf1/7c833b141ad64d17a54fa2a57618d1fc 2023-05-02 02:54:10,855 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf1/5a14c4b25f9c46ed9fea0c2b36698e80 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf1/5a14c4b25f9c46ed9fea0c2b36698e80 2023-05-02 02:54:10,857 DEBUG [HFileArchiver-1] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1, FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2, FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/recovered.edits] 2023-05-02 02:54:10,858 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf2/046985a41802469ba7607724dfd06d98 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/cf2/046985a41802469ba7607724dfd06d98 2023-05-02 02:54:10,858 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf2/847c4a3c095d47c692df905147536de6 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/cf2/847c4a3c095d47c692df905147536de6 2023-05-02 02:54:10,860 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/3189d0758d394d4290511c7d95e297bb to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/3189d0758d394d4290511c7d95e297bb 2023-05-02 02:54:10,861 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/recovered.edits/113.seqid to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68/recovered.edits/113.seqid 2023-05-02 02:54:10,861 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/recovered.edits/113.seqid to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9/recovered.edits/113.seqid 2023-05-02 02:54:10,862 DEBUG [HFileArchiver-5] backup.HFileArchiver(596): Deleted hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/024f19513d3a9ded6c060054f1607e68 2023-05-02 02:54:10,862 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/40f1a5f529284647a6cf211a61001756 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/40f1a5f529284647a6cf211a61001756 2023-05-02 02:54:10,862 DEBUG [HFileArchiver-2] backup.HFileArchiver(596): Deleted hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/bb895e1df53cda46bfcb5f94fe1710c9 2023-05-02 02:54:10,863 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/609652eb416344f8b705e1e96171ce1e to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/609652eb416344f8b705e1e96171ce1e 2023-05-02 02:54:10,864 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/72ce99472ae84721bfaf8e780648de42 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/72ce99472ae84721bfaf8e780648de42 2023-05-02 02:54:10,865 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/8db73eea04c64e878c8c4529b89946df to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/8db73eea04c64e878c8c4529b89946df 2023-05-02 02:54:10,866 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/b0c982e0384d423aa455befed4ca21c3 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/b0c982e0384d423aa455befed4ca21c3 2023-05-02 02:54:10,868 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/d75d441bf5874fe6b71adefb02039137 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/d75d441bf5874fe6b71adefb02039137 2023-05-02 02:54:10,869 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/dc659b190e7744abb3e0fe388448fa31 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/dc659b190e7744abb3e0fe388448fa31 2023-05-02 02:54:10,870 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/f6fbf066cf0a4af6b2ea76697cc904da to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/f6fbf066cf0a4af6b2ea76697cc904da 2023-05-02 02:54:10,871 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/fa00f661973547868b8b686731148b94 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf1/fa00f661973547868b8b686731148b94 2023-05-02 02:54:10,873 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/09bd463709ed4f1e83a4a0d35349012e to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/09bd463709ed4f1e83a4a0d35349012e 2023-05-02 02:54:10,874 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/378b3c3bb4df4380b5f3244397d9fcb2 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/378b3c3bb4df4380b5f3244397d9fcb2 2023-05-02 02:54:10,875 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/76b4e49972d544bdb7a8702299d57cc6 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/76b4e49972d544bdb7a8702299d57cc6 2023-05-02 02:54:10,877 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/8243f44fc750458da547c0947dff42a0 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/8243f44fc750458da547c0947dff42a0 2023-05-02 02:54:10,878 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/90a8fcb9bc36402ab2af45c21a1f4254 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/90a8fcb9bc36402ab2af45c21a1f4254 2023-05-02 02:54:10,879 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/a02bf38bab7c419c8ba5d1fb45d02f4c to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/a02bf38bab7c419c8ba5d1fb45d02f4c 2023-05-02 02:54:10,880 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/e6bbd81d56884f75a2ab34f638d3dc7f to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/e6bbd81d56884f75a2ab34f638d3dc7f 2023-05-02 02:54:10,881 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/eb4bc94f1cf94bc080443cc926ed2a17 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/eb4bc94f1cf94bc080443cc926ed2a17 2023-05-02 02:54:10,882 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/eecb4f6bc8094e669b2ee7981bc5a39f to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/eecb4f6bc8094e669b2ee7981bc5a39f 2023-05-02 02:54:10,883 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/f894abdeb02b4e87bc8789e67b483cc0 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/cf2/f894abdeb02b4e87bc8789e67b483cc0 2023-05-02 02:54:10,886 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/recovered.edits/106.seqid to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f/recovered.edits/106.seqid 2023-05-02 02:54:10,886 DEBUG [HFileArchiver-1] backup.HFileArchiver(596): Deleted hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8af7d4111b21c70efea08f7beab73a9f 2023-05-02 02:54:10,886 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-05-02 02:54:10,888 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=84, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:10,890 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionDeletedRowsDaughter from hbase:meta 2023-05-02 02:54:10,892 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionDeletedRowsDaughter' descriptor. 2023-05-02 02:54:10,893 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=84, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:10,893 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionDeletedRowsDaughter' from region states. 2023-05-02 02:54:10,893 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1682996046681.8af7d4111b21c70efea08f7beab73a9f.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1682996050893"}]},"ts":"9223372036854775807"} 2023-05-02 02:54:10,893 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1682996048869.bb895e1df53cda46bfcb5f94fe1710c9.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1682996050893"}]},"ts":"9223372036854775807"} 2023-05-02 02:54:10,893 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,60,1682996048869.024f19513d3a9ded6c060054f1607e68.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1682996050893"}]},"ts":"9223372036854775807"} 2023-05-02 02:54:10,895 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-05-02 02:54:10,895 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 8af7d4111b21c70efea08f7beab73a9f, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1682996046681.8af7d4111b21c70efea08f7beab73a9f.', STARTKEY => '', ENDKEY => ''}, {ENCODED => bb895e1df53cda46bfcb5f94fe1710c9, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1682996048869.bb895e1df53cda46bfcb5f94fe1710c9.', STARTKEY => '', ENDKEY => '60'}, {ENCODED => 024f19513d3a9ded6c060054f1607e68, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1682996048869.024f19513d3a9ded6c060054f1607e68.', STARTKEY => '60', ENDKEY => ''}] 2023-05-02 02:54:10,895 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionDeletedRowsDaughter' as deleted. 2023-05-02 02:54:10,895 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1682996050895"}]},"ts":"9223372036854775807"} 2023-05-02 02:54:10,896 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionDeletedRowsDaughter state from META 2023-05-02 02:54:10,898 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=84, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:54:10,899 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 56 msec 2023-05-02 02:54:10,947 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1227): Checking to see if procedure is done pid=84 2023-05-02 02:54:10,947 INFO [Listener at localhost/37119] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 84 completed 2023-05-02 02:54:10,965 INFO [Listener at localhost/37119] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=411 (was 406) Potentially hanging thread: hconnection-0x2c67b29-shared-pool-14 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1250645491_15 at /127.0.0.1:60324 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-5 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1250645491_15 at /127.0.0.1:40848 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2c67b29-metaLookup-shared--pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1250645491_15 at /127.0.0.1:59940 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2c67b29-shared-pool-17 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2c67b29-shared-pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1250645491_15 at /127.0.0.1:40772 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1250645491_15 at /127.0.0.1:60310 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1250645491_15 at /127.0.0.1:39370 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2c67b29-metaLookup-shared--pool-14 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1250645491_15 at /127.0.0.1:40804 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2c67b29-metaLookup-shared--pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1250645491_15 at /127.0.0.1:40816 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1250645491_15 at /127.0.0.1:39358 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1250645491_15 at /127.0.0.1:60254 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1250645491_15 at /127.0.0.1:60286 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1250645491_15 at /127.0.0.1:59676 [Waiting for operation #19] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2c67b29-metaLookup-shared--pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1250645491_15 at /127.0.0.1:60294 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2c67b29-shared-pool-16 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2c67b29-metaLookup-shared--pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1250645491_15 at /127.0.0.1:39352 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1250645491_15 at /127.0.0.1:60288 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1250645491_15 at /127.0.0.1:60256 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2c67b29-shared-pool-15 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=869 (was 867) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=254 (was 268), ProcessCount=170 (was 170), AvailableMemoryMB=13838 (was 13852) 2023-05-02 02:54:10,985 INFO [Listener at localhost/37119] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=411, OpenFileDescriptor=869, MaxFileDescriptor=60000, SystemLoadAverage=254, ProcessCount=170, AvailableMemoryMB=13836 2023-05-02 02:54:10,986 WARN [Listener at localhost/37119] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-02 02:54:10,989 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-05-02 02:54:10,989 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-05-02 02:54:10,991 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(1029): Stored pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-02 02:54:10,993 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_PRE_OPERATION 2023-05-02 02:54:10,994 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-05-02 02:54:10,995 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/74b19512a655e6f00fc418d81fba16c8 2023-05-02 02:54:10,996 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/74b19512a655e6f00fc418d81fba16c8 empty. 2023-05-02 02:54:10,996 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/74b19512a655e6f00fc418d81fba16c8 2023-05-02 02:54:10,996 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-05-02 02:54:11,009 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-05-02 02:54:11,010 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(7675): creating {ENCODED => 74b19512a655e6f00fc418d81fba16c8, NAME => 'testRollbackForSplitTableRegionWithReplica,,1682996050990.74b19512a655e6f00fc418d81fba16c8.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp 2023-05-02 02:54:11,023 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(866): Instantiated testRollbackForSplitTableRegionWithReplica,,1682996050990.74b19512a655e6f00fc418d81fba16c8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-02 02:54:11,023 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1604): Closing 74b19512a655e6f00fc418d81fba16c8, disabling compactions & flushes 2023-05-02 02:54:11,023 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1626): Closing region testRollbackForSplitTableRegionWithReplica,,1682996050990.74b19512a655e6f00fc418d81fba16c8. 2023-05-02 02:54:11,023 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1682996050990.74b19512a655e6f00fc418d81fba16c8. 2023-05-02 02:54:11,023 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1714): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1682996050990.74b19512a655e6f00fc418d81fba16c8. after waiting 0 ms 2023-05-02 02:54:11,023 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1724): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1682996050990.74b19512a655e6f00fc418d81fba16c8. 2023-05-02 02:54:11,023 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1838): Closed testRollbackForSplitTableRegionWithReplica,,1682996050990.74b19512a655e6f00fc418d81fba16c8. 2023-05-02 02:54:11,023 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1558): Region close journal for 74b19512a655e6f00fc418d81fba16c8: 2023-05-02 02:54:11,025 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ADD_TO_META 2023-05-02 02:54:11,026 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1682996050990.74b19512a655e6f00fc418d81fba16c8.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1682996051026"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682996051026"}]},"ts":"1682996051026"} 2023-05-02 02:54:11,028 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-05-02 02:54:11,029 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-05-02 02:54:11,029 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682996051029"}]},"ts":"1682996051029"} 2023-05-02 02:54:11,030 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLING in hbase:meta 2023-05-02 02:54:11,032 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-05-02 02:54:11,033 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-05-02 02:54:11,033 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-05-02 02:54:11,033 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-05-02 02:54:11,033 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-05-02 02:54:11,034 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=74b19512a655e6f00fc418d81fba16c8, ASSIGN}] 2023-05-02 02:54:11,035 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=74b19512a655e6f00fc418d81fba16c8, ASSIGN 2023-05-02 02:54:11,036 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=74b19512a655e6f00fc418d81fba16c8, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42019,1682996021676; forceNewPlan=false, retain=false 2023-05-02 02:54:11,186 INFO [jenkins-hbase4:41119] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-05-02 02:54:11,189 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=74b19512a655e6f00fc418d81fba16c8, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:11,189 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1682996050990.74b19512a655e6f00fc418d81fba16c8.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1682996051189"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1682996051189"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682996051189"}]},"ts":"1682996051189"} 2023-05-02 02:54:11,191 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=87, ppid=86, state=RUNNABLE; OpenRegionProcedure 74b19512a655e6f00fc418d81fba16c8, server=jenkins-hbase4.apache.org,42019,1682996021676}] 2023-05-02 02:54:11,345 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRollbackForSplitTableRegionWithReplica,,1682996050990.74b19512a655e6f00fc418d81fba16c8. 2023-05-02 02:54:11,345 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 74b19512a655e6f00fc418d81fba16c8, NAME => 'testRollbackForSplitTableRegionWithReplica,,1682996050990.74b19512a655e6f00fc418d81fba16c8.', STARTKEY => '', ENDKEY => ''} 2023-05-02 02:54:11,346 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-02 02:54:11,346 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 74b19512a655e6f00fc418d81fba16c8 2023-05-02 02:54:11,346 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRollbackForSplitTableRegionWithReplica,,1682996050990.74b19512a655e6f00fc418d81fba16c8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-02 02:54:11,346 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 74b19512a655e6f00fc418d81fba16c8 2023-05-02 02:54:11,346 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 74b19512a655e6f00fc418d81fba16c8 2023-05-02 02:54:11,347 INFO [StoreOpener-74b19512a655e6f00fc418d81fba16c8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 74b19512a655e6f00fc418d81fba16c8 2023-05-02 02:54:11,348 DEBUG [StoreOpener-74b19512a655e6f00fc418d81fba16c8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackForSplitTableRegionWithReplica/74b19512a655e6f00fc418d81fba16c8/cf1 2023-05-02 02:54:11,348 DEBUG [StoreOpener-74b19512a655e6f00fc418d81fba16c8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackForSplitTableRegionWithReplica/74b19512a655e6f00fc418d81fba16c8/cf1 2023-05-02 02:54:11,349 INFO [StoreOpener-74b19512a655e6f00fc418d81fba16c8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 74b19512a655e6f00fc418d81fba16c8 columnFamilyName cf1 2023-05-02 02:54:11,349 INFO [StoreOpener-74b19512a655e6f00fc418d81fba16c8-1] regionserver.HStore(310): Store=74b19512a655e6f00fc418d81fba16c8/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-02 02:54:11,350 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackForSplitTableRegionWithReplica/74b19512a655e6f00fc418d81fba16c8 2023-05-02 02:54:11,350 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackForSplitTableRegionWithReplica/74b19512a655e6f00fc418d81fba16c8 2023-05-02 02:54:11,352 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 74b19512a655e6f00fc418d81fba16c8 2023-05-02 02:54:11,354 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackForSplitTableRegionWithReplica/74b19512a655e6f00fc418d81fba16c8/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-05-02 02:54:11,354 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 74b19512a655e6f00fc418d81fba16c8; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64687078, jitterRate=-0.03608742356300354}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-05-02 02:54:11,354 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 74b19512a655e6f00fc418d81fba16c8: 2023-05-02 02:54:11,355 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1682996050990.74b19512a655e6f00fc418d81fba16c8., pid=87, masterSystemTime=1682996051342 2023-05-02 02:54:11,357 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1682996050990.74b19512a655e6f00fc418d81fba16c8. 2023-05-02 02:54:11,357 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRollbackForSplitTableRegionWithReplica,,1682996050990.74b19512a655e6f00fc418d81fba16c8. 2023-05-02 02:54:11,357 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=74b19512a655e6f00fc418d81fba16c8, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:11,357 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1682996050990.74b19512a655e6f00fc418d81fba16c8.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1682996051357"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1682996051357"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1682996051357"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682996051357"}]},"ts":"1682996051357"} 2023-05-02 02:54:11,359 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=87, resume processing ppid=86 2023-05-02 02:54:11,359 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=87, ppid=86, state=SUCCESS; OpenRegionProcedure 74b19512a655e6f00fc418d81fba16c8, server=jenkins-hbase4.apache.org,42019,1682996021676 in 167 msec 2023-05-02 02:54:11,361 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=86, resume processing ppid=85 2023-05-02 02:54:11,361 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=86, ppid=85, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=74b19512a655e6f00fc418d81fba16c8, ASSIGN in 326 msec 2023-05-02 02:54:11,361 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-05-02 02:54:11,361 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682996051361"}]},"ts":"1682996051361"} 2023-05-02 02:54:11,362 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLED in hbase:meta 2023-05-02 02:54:11,370 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_POST_OPERATION 2023-05-02 02:54:11,371 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica in 380 msec 2023-05-02 02:54:11,501 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testRollbackForSplitTableRegionWithReplica from 'testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} to 'testRollbackForSplitTableRegionWithReplica', {TABLE_ATTRIBUTES => {REGION_REPLICATION => '2'}}, {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-05-02 02:54:11,510 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure2.ProcedureExecutor(1029): Stored pid=88, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-02 02:54:11,528 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-05-02 02:54:11,529 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-05-02 02:54:11,529 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-05-02 02:54:11,532 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=89, ppid=88, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica}] 2023-05-02 02:54:11,543 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=74b19512a655e6f00fc418d81fba16c8, REOPEN/MOVE}] 2023-05-02 02:54:11,545 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=74b19512a655e6f00fc418d81fba16c8, REOPEN/MOVE 2023-05-02 02:54:11,545 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=74b19512a655e6f00fc418d81fba16c8, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:11,545 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1682996050990.74b19512a655e6f00fc418d81fba16c8.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1682996051545"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1682996051545"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682996051545"}]},"ts":"1682996051545"} 2023-05-02 02:54:11,546 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=91, ppid=90, state=RUNNABLE; CloseRegionProcedure 74b19512a655e6f00fc418d81fba16c8, server=jenkins-hbase4.apache.org,42019,1682996021676}] 2023-05-02 02:54:11,698 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 74b19512a655e6f00fc418d81fba16c8 2023-05-02 02:54:11,699 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 74b19512a655e6f00fc418d81fba16c8, disabling compactions & flushes 2023-05-02 02:54:11,699 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRollbackForSplitTableRegionWithReplica,,1682996050990.74b19512a655e6f00fc418d81fba16c8. 2023-05-02 02:54:11,700 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1682996050990.74b19512a655e6f00fc418d81fba16c8. 2023-05-02 02:54:11,700 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1682996050990.74b19512a655e6f00fc418d81fba16c8. after waiting 0 ms 2023-05-02 02:54:11,700 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1682996050990.74b19512a655e6f00fc418d81fba16c8. 2023-05-02 02:54:11,703 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackForSplitTableRegionWithReplica/74b19512a655e6f00fc418d81fba16c8/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-05-02 02:54:11,703 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-02 02:54:11,703 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRollbackForSplitTableRegionWithReplica,,1682996050990.74b19512a655e6f00fc418d81fba16c8. 2023-05-02 02:54:11,703 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 74b19512a655e6f00fc418d81fba16c8: 2023-05-02 02:54:11,703 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 74b19512a655e6f00fc418d81fba16c8 to self. 2023-05-02 02:54:11,705 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 74b19512a655e6f00fc418d81fba16c8 2023-05-02 02:54:11,705 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=74b19512a655e6f00fc418d81fba16c8, regionState=CLOSED 2023-05-02 02:54:11,705 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1682996050990.74b19512a655e6f00fc418d81fba16c8.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1682996051705"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682996051705"}]},"ts":"1682996051705"} 2023-05-02 02:54:11,708 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=91, resume processing ppid=90 2023-05-02 02:54:11,708 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=91, ppid=90, state=SUCCESS; CloseRegionProcedure 74b19512a655e6f00fc418d81fba16c8, server=jenkins-hbase4.apache.org,42019,1682996021676 in 161 msec 2023-05-02 02:54:11,709 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=74b19512a655e6f00fc418d81fba16c8, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,42019,1682996021676; forceNewPlan=false, retain=true 2023-05-02 02:54:11,859 INFO [jenkins-hbase4:41119] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-05-02 02:54:11,859 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=74b19512a655e6f00fc418d81fba16c8, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:11,859 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1682996050990.74b19512a655e6f00fc418d81fba16c8.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1682996051859"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1682996051859"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682996051859"}]},"ts":"1682996051859"} 2023-05-02 02:54:11,861 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=92, ppid=90, state=RUNNABLE; OpenRegionProcedure 74b19512a655e6f00fc418d81fba16c8, server=jenkins-hbase4.apache.org,42019,1682996021676}] 2023-05-02 02:54:12,016 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRollbackForSplitTableRegionWithReplica,,1682996050990.74b19512a655e6f00fc418d81fba16c8. 2023-05-02 02:54:12,016 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 74b19512a655e6f00fc418d81fba16c8, NAME => 'testRollbackForSplitTableRegionWithReplica,,1682996050990.74b19512a655e6f00fc418d81fba16c8.', STARTKEY => '', ENDKEY => ''} 2023-05-02 02:54:12,016 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-02 02:54:12,016 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 74b19512a655e6f00fc418d81fba16c8 2023-05-02 02:54:12,016 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRollbackForSplitTableRegionWithReplica,,1682996050990.74b19512a655e6f00fc418d81fba16c8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-02 02:54:12,016 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 74b19512a655e6f00fc418d81fba16c8 2023-05-02 02:54:12,016 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 74b19512a655e6f00fc418d81fba16c8 2023-05-02 02:54:12,017 INFO [StoreOpener-74b19512a655e6f00fc418d81fba16c8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 74b19512a655e6f00fc418d81fba16c8 2023-05-02 02:54:12,018 DEBUG [StoreOpener-74b19512a655e6f00fc418d81fba16c8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackForSplitTableRegionWithReplica/74b19512a655e6f00fc418d81fba16c8/cf1 2023-05-02 02:54:12,018 DEBUG [StoreOpener-74b19512a655e6f00fc418d81fba16c8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackForSplitTableRegionWithReplica/74b19512a655e6f00fc418d81fba16c8/cf1 2023-05-02 02:54:12,019 INFO [StoreOpener-74b19512a655e6f00fc418d81fba16c8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 74b19512a655e6f00fc418d81fba16c8 columnFamilyName cf1 2023-05-02 02:54:12,019 INFO [StoreOpener-74b19512a655e6f00fc418d81fba16c8-1] regionserver.HStore(310): Store=74b19512a655e6f00fc418d81fba16c8/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-02 02:54:12,020 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackForSplitTableRegionWithReplica/74b19512a655e6f00fc418d81fba16c8 2023-05-02 02:54:12,021 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackForSplitTableRegionWithReplica/74b19512a655e6f00fc418d81fba16c8 2023-05-02 02:54:12,023 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 74b19512a655e6f00fc418d81fba16c8 2023-05-02 02:54:12,023 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 74b19512a655e6f00fc418d81fba16c8; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64394100, jitterRate=-0.04045313596725464}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-05-02 02:54:12,023 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 74b19512a655e6f00fc418d81fba16c8: 2023-05-02 02:54:12,024 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1682996050990.74b19512a655e6f00fc418d81fba16c8., pid=92, masterSystemTime=1682996052013 2023-05-02 02:54:12,025 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1682996050990.74b19512a655e6f00fc418d81fba16c8. 2023-05-02 02:54:12,025 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRollbackForSplitTableRegionWithReplica,,1682996050990.74b19512a655e6f00fc418d81fba16c8. 2023-05-02 02:54:12,026 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=74b19512a655e6f00fc418d81fba16c8, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:54:12,026 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1682996050990.74b19512a655e6f00fc418d81fba16c8.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1682996052026"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1682996052026"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1682996052026"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682996052026"}]},"ts":"1682996052026"} 2023-05-02 02:54:12,028 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=92, resume processing ppid=90 2023-05-02 02:54:12,029 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=92, ppid=90, state=SUCCESS; OpenRegionProcedure 74b19512a655e6f00fc418d81fba16c8, server=jenkins-hbase4.apache.org,42019,1682996021676 in 166 msec 2023-05-02 02:54:12,030 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=90, resume processing ppid=89 2023-05-02 02:54:12,030 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=90, ppid=89, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=74b19512a655e6f00fc418d81fba16c8, REOPEN/MOVE in 486 msec 2023-05-02 02:54:12,032 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=89, resume processing ppid=88 2023-05-02 02:54:12,032 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=89, ppid=88, state=SUCCESS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica in 500 msec 2023-05-02 02:54:12,035 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=1b91efe506b89bc57ba68287b021be4f, ASSIGN}] 2023-05-02 02:54:12,036 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=1b91efe506b89bc57ba68287b021be4f, ASSIGN 2023-05-02 02:54:12,037 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=1b91efe506b89bc57ba68287b021be4f, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-05-02 02:54:12,187 DEBUG [jenkins-hbase4:41119] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-05-02 02:54:12,188 DEBUG [jenkins-hbase4:41119] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-05-02 02:54:12,188 DEBUG [jenkins-hbase4:41119] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-05-02 02:54:12,188 DEBUG [jenkins-hbase4:41119] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-05-02 02:54:12,188 DEBUG [jenkins-hbase4:41119] balancer.BaseLoadBalancer$Cluster(378): Number of tables=3, number of hosts=1, number of racks=1 2023-05-02 02:54:12,190 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=1b91efe506b89bc57ba68287b021be4f, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:54:12,190 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1682996050990.74b19512a655e6f00fc418d81fba16c8.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1682996052190"},{"qualifier":"sn_0001","vlen":45,"tag":[],"timestamp":"1682996052190"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1682996052190"}]},"ts":"1682996052190"} 2023-05-02 02:54:12,194 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=94, ppid=93, state=RUNNABLE; OpenRegionProcedure 1b91efe506b89bc57ba68287b021be4f, server=jenkins-hbase4.apache.org,39081,1682996021579}] 2023-05-02 02:54:12,349 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRollbackForSplitTableRegionWithReplica,,1682996050990_0001.1b91efe506b89bc57ba68287b021be4f. 2023-05-02 02:54:12,349 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1b91efe506b89bc57ba68287b021be4f, NAME => 'testRollbackForSplitTableRegionWithReplica,,1682996050990_0001.1b91efe506b89bc57ba68287b021be4f.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1} 2023-05-02 02:54:12,349 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-02 02:54:12,350 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 1b91efe506b89bc57ba68287b021be4f 2023-05-02 02:54:12,350 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRollbackForSplitTableRegionWithReplica,,1682996050990_0001.1b91efe506b89bc57ba68287b021be4f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-02 02:54:12,350 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1b91efe506b89bc57ba68287b021be4f 2023-05-02 02:54:12,350 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1b91efe506b89bc57ba68287b021be4f 2023-05-02 02:54:12,350 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:12,450 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:12,550 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:12,651 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:12,751 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:12,851 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:12,951 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:13,051 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:13,152 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:13,252 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:13,297 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackForSplitTableRegionWithReplica' 2023-05-02 02:54:13,352 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:13,452 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:13,552 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:13,653 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:13,753 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:13,853 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:13,953 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:14,053 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:14,154 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:14,254 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:14,354 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:14,454 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:14,554 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:14,655 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:14,755 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:14,855 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:14,955 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:15,056 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:15,156 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:15,256 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:15,356 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:15,456 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:15,554 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-05-02 02:54:15,556 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:15,657 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:15,757 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:15,857 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:15,957 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:16,057 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:16,158 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:16,258 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:16,358 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:16,458 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:16,558 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:16,659 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:16,759 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:16,859 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:16,959 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:17,059 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:17,160 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:17,260 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:17,360 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:17,460 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:17,560 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:17,661 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:17,761 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:17,861 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:17,961 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:18,061 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:18,162 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:18,262 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:18,362 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:18,462 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:18,562 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:18,663 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:18,763 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:18,863 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:18,963 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:19,064 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:19,164 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:19,264 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:19,364 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:19,464 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:19,565 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:19,665 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:19,765 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:19,865 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:19,965 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:20,066 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:20,166 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:20,266 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:20,366 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:20,466 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:20,566 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:20,667 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:20,767 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:20,867 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:20,967 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:21,068 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:21,168 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:21,268 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:21,368 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:21,468 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:21,569 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:21,669 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:21,769 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:21,869 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:21,969 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:22,070 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:22,170 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:22,270 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:22,370 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:22,470 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:22,571 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:22,671 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:22,771 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:22,871 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:22,971 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:23,072 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:23,172 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:23,272 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:23,372 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:23,472 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:23,573 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:23,673 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:23,773 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:23,873 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:23,973 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:24,074 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:24,174 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:24,274 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:24,374 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:24,474 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:24,575 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:24,675 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:24,775 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:24,875 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:24,975 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:25,076 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:25,176 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:25,276 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:25,376 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:25,477 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:25,577 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:25,677 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:25,777 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:25,877 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:25,978 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:26,078 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:26,178 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:26,278 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:26,378 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:26,479 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:26,579 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:26,679 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:26,779 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:26,879 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:26,980 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:27,080 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:27,180 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:27,280 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:27,380 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:27,481 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:27,581 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:27,681 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:27,781 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:27,882 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:27,982 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:28,082 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:28,182 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:28,282 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:28,383 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:28,483 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:28,583 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:28,683 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:28,783 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:28,884 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:28,984 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:29,084 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:29,184 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:29,284 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:29,385 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:29,485 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:29,585 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:29,685 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:29,785 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:29,886 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:29,986 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:30,086 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:30,186 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:30,286 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:30,387 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:30,487 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:30,587 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:30,687 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:30,787 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:30,888 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:30,988 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:31,088 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:31,188 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:31,288 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:31,388 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:31,489 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:31,589 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:31,689 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:31,789 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:31,890 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:31,990 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:32,090 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:32,190 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:32,290 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:32,391 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:32,491 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:32,591 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:32,691 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:32,791 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:32,892 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:32,992 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:33,092 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:33,192 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:33,292 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:33,393 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:33,493 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:33,593 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:33,693 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:33,793 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:33,894 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:33,994 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:34,094 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:34,194 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:34,296 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:34,396 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:34,496 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:34,597 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:34,697 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:34,797 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:34,897 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:34,997 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:35,097 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:35,198 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:35,298 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:35,398 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:35,498 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:35,599 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:35,699 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:35,799 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:35,899 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:35,999 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:36,100 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:36,200 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:36,300 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:36,400 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:36,500 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:36,601 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:36,701 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:36,801 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:36,901 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:37,001 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:37,102 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:37,202 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:37,302 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:37,402 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:37,502 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:37,603 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:37,703 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:37,803 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:37,903 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:38,003 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:38,104 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:38,204 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:38,304 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:38,404 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:38,504 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:38,605 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:38,705 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:38,805 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:38,905 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:39,005 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:39,106 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:39,206 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:39,306 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:39,406 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:39,506 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:39,607 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:39,707 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:39,807 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:39,907 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:40,008 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:40,108 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:40,208 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:40,308 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:40,409 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:40,509 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:40,609 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:40,709 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:40,809 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:40,909 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:41,010 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:41,110 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:41,210 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:41,310 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:41,411 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:41,511 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:41,611 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:41,711 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:41,811 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:41,912 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:42,012 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:42,112 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:42,212 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:42,312 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:42,413 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:42,513 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:42,613 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:42,713 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:42,813 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:42,914 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:43,014 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:43,114 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:43,214 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:43,314 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:43,415 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:43,515 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:43,615 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:43,715 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:43,815 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:43,916 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:44,016 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:44,116 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:44,216 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:44,316 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:44,417 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:44,517 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:44,617 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:44,717 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:44,817 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:44,918 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:45,018 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:45,118 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:45,218 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:45,318 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:45,419 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:45,519 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:45,619 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:45,719 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:45,819 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:45,920 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:46,020 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:46,120 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:46,220 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:46,320 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:46,421 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:46,521 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:46,621 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:46,721 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:46,821 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:46,922 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:47,022 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:47,122 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:47,222 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:47,322 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:47,423 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:47,523 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:47,623 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:47,723 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:47,823 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:47,924 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:48,024 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:48,124 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:48,224 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:48,324 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:48,425 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:48,525 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:48,625 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:48,725 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:48,825 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:48,926 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:49,026 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:49,126 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:49,226 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:49,326 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:49,427 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:49,527 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:49,627 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:49,727 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:49,827 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:49,928 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:50,028 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:50,128 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:50,228 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:50,328 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:50,429 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:50,529 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:50,629 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:50,729 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:50,829 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:50,930 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:51,030 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:51,130 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:51,230 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:51,330 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:51,431 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:51,531 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:51,631 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:51,731 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:51,831 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:51,932 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:52,032 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:52,132 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:52,232 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:52,333 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:52,433 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:52,533 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:52,633 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:52,733 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:52,834 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:52,934 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:53,034 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:53,134 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:53,235 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:53,335 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:53,435 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:53,535 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:53,635 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:53,736 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:53,836 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:53,936 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:54,036 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:54,136 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:54,237 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:54,337 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:54,437 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:54,537 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:54,637 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:54,738 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:54,838 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:54,938 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:55,038 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:55,138 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:55,239 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:55,339 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:55,439 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:55,539 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:55,639 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:55,740 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:55,840 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:55,940 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:56,040 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:56,140 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:56,241 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:56,341 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:56,441 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:56,541 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:56,641 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:56,742 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:56,842 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:56,942 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:57,042 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:57,143 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:57,243 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:57,343 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:57,443 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:57,543 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:57,644 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:57,744 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:57,844 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:57,944 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:58,044 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:58,145 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:58,245 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:58,345 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:58,445 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:58,545 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:58,646 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:58,746 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:58,846 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:58,946 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:59,046 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:59,147 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:59,247 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:59,347 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:59,447 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:59,547 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:59,648 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:59,748 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:59,848 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:54:59,948 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:00,049 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:00,149 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:00,249 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:00,349 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:00,449 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:00,550 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:00,650 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:00,750 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:00,850 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:00,950 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:01,051 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:01,151 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:01,251 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:01,351 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:01,452 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:01,552 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:01,652 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:01,752 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:01,852 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:01,953 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:02,053 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:02,153 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:02,253 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:02,353 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:02,454 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:02,554 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:02,654 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:02,754 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:02,854 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:02,955 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:03,055 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:03,155 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:03,255 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:03,355 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:03,456 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:03,556 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:03,656 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:03,756 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:03,856 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:03,957 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:04,057 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:04,157 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:04,257 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:04,357 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:04,458 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:04,558 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:04,658 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:04,758 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:04,858 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:04,959 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:05,059 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:05,159 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:05,259 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:05,359 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:05,460 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:05,560 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:05,660 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:05,760 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:05,861 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:05,961 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:06,061 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:06,161 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:06,261 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:06,362 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:06,462 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:06,562 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:06,662 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:06,762 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:06,863 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:06,963 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:07,063 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:07,163 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:07,263 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:07,364 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:07,464 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:07,564 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:07,664 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:07,765 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:07,865 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:07,965 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:08,065 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:08,165 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:08,266 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:08,366 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:08,466 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:08,566 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:08,666 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:08,767 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:08,867 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:08,967 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:09,067 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:09,167 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:09,268 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:09,368 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:09,468 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:09,568 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:09,668 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:09,769 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:09,869 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:09,969 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:10,069 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:10,169 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:10,270 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:10,370 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:10,470 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:10,570 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:10,670 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:10,771 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:10,871 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:10,971 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:11,071 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:11,172 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:11,272 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:11,372 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:11,472 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:11,572 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:11,620 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41119] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-05-02 02:55:11,673 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:11,721 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41119] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-05-02 02:55:11,773 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:11,873 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:11,922 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41119] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-05-02 02:55:11,973 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:12,073 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:12,174 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:12,223 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41119] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-05-02 02:55:12,274 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:12,374 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:12,475 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:12,575 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:12,675 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:12,724 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41119] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-05-02 02:55:12,775 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:12,875 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:12,976 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:13,076 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:13,176 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:13,276 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:13,377 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:13,477 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:13,577 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:13,677 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:13,725 DEBUG [Listener at localhost/37119] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:13,726 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(1029): Stored pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=74b19512a655e6f00fc418d81fba16c8, daughterA=bfa227908c6548a4187e4deb87e808a6, daughterB=da50a5f8e634bd92f99eebe500206397 2023-05-02 02:55:13,727 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=74b19512a655e6f00fc418d81fba16c8, daughterA=bfa227908c6548a4187e4deb87e808a6, daughterB=da50a5f8e634bd92f99eebe500206397 2023-05-02 02:55:13,728 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=74b19512a655e6f00fc418d81fba16c8, daughterA=bfa227908c6548a4187e4deb87e808a6, daughterB=da50a5f8e634bd92f99eebe500206397 2023-05-02 02:55:13,728 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=74b19512a655e6f00fc418d81fba16c8, daughterA=bfa227908c6548a4187e4deb87e808a6, daughterB=da50a5f8e634bd92f99eebe500206397 2023-05-02 02:55:13,729 ERROR [PEWorker-1] assignment.SplitTableRegionProcedure(347): Splitting 74b19512a655e6f00fc418d81fba16c8, pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=74b19512a655e6f00fc418d81fba16c8, daughterA=bfa227908c6548a4187e4deb87e808a6, daughterB=da50a5f8e634bd92f99eebe500206397 org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,39081,1682996021579, table=testRollbackForSplitTableRegionWithReplica, region=1b91efe506b89bc57ba68287b021be4f is currently in transition, give up at org.apache.hadoop.hbase.master.assignment.AssignmentManagerUtil.createUnassignProceduresForSplitOrMerge(AssignmentManagerUtil.java:120) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.createUnassignProcedures(SplitTableRegionProcedure.java:881) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:298) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:94) at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188) at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:922) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1646) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-02 02:55:13,732 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,39081,1682996021579, table=testRollbackForSplitTableRegionWithReplica, region=1b91efe506b89bc57ba68287b021be4f is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=74b19512a655e6f00fc418d81fba16c8, daughterA=bfa227908c6548a4187e4deb87e808a6, daughterB=da50a5f8e634bd92f99eebe500206397 exec-time=6 msec 2023-05-02 02:55:13,777 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:13,878 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-02 02:55:13,978 INFO [StoreOpener-1b91efe506b89bc57ba68287b021be4f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 1b91efe506b89bc57ba68287b021be4f 2023-05-02 02:55:13,981 DEBUG [StoreOpener-1b91efe506b89bc57ba68287b021be4f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackForSplitTableRegionWithReplica/74b19512a655e6f00fc418d81fba16c8/cf1 2023-05-02 02:55:13,982 DEBUG [StoreOpener-1b91efe506b89bc57ba68287b021be4f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackForSplitTableRegionWithReplica/74b19512a655e6f00fc418d81fba16c8/cf1 2023-05-02 02:55:13,982 INFO [StoreOpener-1b91efe506b89bc57ba68287b021be4f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1b91efe506b89bc57ba68287b021be4f columnFamilyName cf1 2023-05-02 02:55:13,983 INFO [StoreOpener-1b91efe506b89bc57ba68287b021be4f-1] regionserver.HStore(310): Store=1b91efe506b89bc57ba68287b021be4f/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-02 02:55:13,984 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1b91efe506b89bc57ba68287b021be4f; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=75497442, jitterRate=0.12499955296516418}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-05-02 02:55:13,984 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1b91efe506b89bc57ba68287b021be4f: 2023-05-02 02:55:13,985 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1682996050990_0001.1b91efe506b89bc57ba68287b021be4f., pid=94, masterSystemTime=1682996052346 2023-05-02 02:55:13,986 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1682996050990_0001.1b91efe506b89bc57ba68287b021be4f. 2023-05-02 02:55:13,986 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRollbackForSplitTableRegionWithReplica,,1682996050990_0001.1b91efe506b89bc57ba68287b021be4f. 2023-05-02 02:55:13,987 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=1b91efe506b89bc57ba68287b021be4f, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:13,987 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1682996050990.74b19512a655e6f00fc418d81fba16c8.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1682996113986"},{"qualifier":"server_0001","vlen":31,"tag":[],"timestamp":"1682996113986"},{"qualifier":"serverstartcode_0001","vlen":8,"tag":[],"timestamp":"1682996113986"},{"qualifier":"seqnumDuringOpen_0001","vlen":8,"tag":[],"timestamp":"1682996113986"}]},"ts":"1682996113986"} 2023-05-02 02:55:13,990 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=94, resume processing ppid=93 2023-05-02 02:55:13,990 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=94, ppid=93, state=SUCCESS; OpenRegionProcedure 1b91efe506b89bc57ba68287b021be4f, server=jenkins-hbase4.apache.org,39081,1682996021579 in 1 mins, 1.796 sec 2023-05-02 02:55:13,991 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=93, resume processing ppid=88 2023-05-02 02:55:13,992 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=93, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=1b91efe506b89bc57ba68287b021be4f, ASSIGN in 1 mins, 1.955 sec 2023-05-02 02:55:13,994 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica in 1 mins, 2.488 sec 2023-05-02 02:55:13,994 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] ipc.RpcServer(528): (responseTooSlow): {"call":"ModifyTable(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest)","starttimems":"1682996051497","responsesize":"2","method":"ModifyTable","param":"TODO: class org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest","processingtimems":62497,"client":"172.31.14.131:34000","queuetimems":0,"class":"HMaster"} 2023-05-02 02:55:13,994 DEBUG [RPCClient-NioEventLoopGroup-6-2] ipc.NettyRpcDuplexHandler(156): Unknown callId: 601, skipping over this response of 3 bytes 2023-05-02 02:55:15,978 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(335): procId=95 exception: org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,39081,1682996021579, table=testRollbackForSplitTableRegionWithReplica, region=1b91efe506b89bc57ba68287b021be4f is currently in transition, give up 2023-05-02 02:55:15,979 WARN [Listener at localhost/37119] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-02 02:55:15,981 INFO [Listener at localhost/37119] client.HBaseAdmin$15(890): Started disable of testRollbackForSplitTableRegionWithReplica 2023-05-02 02:55:15,981 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testRollbackForSplitTableRegionWithReplica 2023-05-02 02:55:15,982 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure2.ProcedureExecutor(1029): Stored pid=96, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-02 02:55:15,984 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-05-02 02:55:15,984 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682996115984"}]},"ts":"1682996115984"} 2023-05-02 02:55:15,985 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLING in hbase:meta 2023-05-02 02:55:15,993 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackForSplitTableRegionWithReplica to state=DISABLING 2023-05-02 02:55:15,993 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=74b19512a655e6f00fc418d81fba16c8, UNASSIGN}, {pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=1b91efe506b89bc57ba68287b021be4f, UNASSIGN}] 2023-05-02 02:55:15,994 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=1b91efe506b89bc57ba68287b021be4f, UNASSIGN 2023-05-02 02:55:15,994 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=1b91efe506b89bc57ba68287b021be4f, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:15,995 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1682996050990.74b19512a655e6f00fc418d81fba16c8.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1682996115994"},{"qualifier":"sn_0001","vlen":45,"tag":[],"timestamp":"1682996115994"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1682996115994"}]},"ts":"1682996115994"} 2023-05-02 02:55:15,996 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=99, ppid=98, state=RUNNABLE; CloseRegionProcedure 1b91efe506b89bc57ba68287b021be4f, server=jenkins-hbase4.apache.org,39081,1682996021579}] 2023-05-02 02:55:15,997 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=74b19512a655e6f00fc418d81fba16c8, UNASSIGN 2023-05-02 02:55:15,998 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=74b19512a655e6f00fc418d81fba16c8, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:15,998 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1682996050990.74b19512a655e6f00fc418d81fba16c8.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1682996115998"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1682996115998"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682996115998"}]},"ts":"1682996115998"} 2023-05-02 02:55:15,999 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=100, ppid=97, state=RUNNABLE; CloseRegionProcedure 74b19512a655e6f00fc418d81fba16c8, server=jenkins-hbase4.apache.org,42019,1682996021676}] 2023-05-02 02:55:16,085 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-05-02 02:55:16,147 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 1b91efe506b89bc57ba68287b021be4f 2023-05-02 02:55:16,148 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1b91efe506b89bc57ba68287b021be4f, disabling compactions & flushes 2023-05-02 02:55:16,149 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRollbackForSplitTableRegionWithReplica,,1682996050990_0001.1b91efe506b89bc57ba68287b021be4f. 2023-05-02 02:55:16,149 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1682996050990_0001.1b91efe506b89bc57ba68287b021be4f. 2023-05-02 02:55:16,149 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1682996050990_0001.1b91efe506b89bc57ba68287b021be4f. after waiting 0 ms 2023-05-02 02:55:16,149 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1682996050990_0001.1b91efe506b89bc57ba68287b021be4f. 2023-05-02 02:55:16,149 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-02 02:55:16,150 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRollbackForSplitTableRegionWithReplica,,1682996050990_0001.1b91efe506b89bc57ba68287b021be4f. 2023-05-02 02:55:16,150 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1b91efe506b89bc57ba68287b021be4f: 2023-05-02 02:55:16,151 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 74b19512a655e6f00fc418d81fba16c8 2023-05-02 02:55:16,151 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 74b19512a655e6f00fc418d81fba16c8, disabling compactions & flushes 2023-05-02 02:55:16,151 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRollbackForSplitTableRegionWithReplica,,1682996050990.74b19512a655e6f00fc418d81fba16c8. 2023-05-02 02:55:16,151 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1682996050990.74b19512a655e6f00fc418d81fba16c8. 2023-05-02 02:55:16,151 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1682996050990.74b19512a655e6f00fc418d81fba16c8. after waiting 0 ms 2023-05-02 02:55:16,152 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1682996050990.74b19512a655e6f00fc418d81fba16c8. 2023-05-02 02:55:16,152 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 1b91efe506b89bc57ba68287b021be4f 2023-05-02 02:55:16,152 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=1b91efe506b89bc57ba68287b021be4f, regionState=CLOSED 2023-05-02 02:55:16,154 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1682996050990.74b19512a655e6f00fc418d81fba16c8.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1682996116152"},{"qualifier":"state_0001","vlen":6,"tag":[],"timestamp":"1682996116152"}]},"ts":"1682996116152"} 2023-05-02 02:55:16,157 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRollbackForSplitTableRegionWithReplica/74b19512a655e6f00fc418d81fba16c8/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-05-02 02:55:16,158 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-02 02:55:16,158 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRollbackForSplitTableRegionWithReplica,,1682996050990.74b19512a655e6f00fc418d81fba16c8. 2023-05-02 02:55:16,158 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 74b19512a655e6f00fc418d81fba16c8: 2023-05-02 02:55:16,159 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 74b19512a655e6f00fc418d81fba16c8 2023-05-02 02:55:16,160 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=99, resume processing ppid=98 2023-05-02 02:55:16,160 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=99, ppid=98, state=SUCCESS; CloseRegionProcedure 1b91efe506b89bc57ba68287b021be4f, server=jenkins-hbase4.apache.org,39081,1682996021579 in 163 msec 2023-05-02 02:55:16,161 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=98, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=1b91efe506b89bc57ba68287b021be4f, UNASSIGN in 167 msec 2023-05-02 02:55:16,162 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=74b19512a655e6f00fc418d81fba16c8, regionState=CLOSED 2023-05-02 02:55:16,162 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1682996050990.74b19512a655e6f00fc418d81fba16c8.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1682996116162"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682996116162"}]},"ts":"1682996116162"} 2023-05-02 02:55:16,171 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=100, resume processing ppid=97 2023-05-02 02:55:16,171 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=100, ppid=97, state=SUCCESS; CloseRegionProcedure 74b19512a655e6f00fc418d81fba16c8, server=jenkins-hbase4.apache.org,42019,1682996021676 in 164 msec 2023-05-02 02:55:16,172 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=97, resume processing ppid=96 2023-05-02 02:55:16,173 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=97, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=74b19512a655e6f00fc418d81fba16c8, UNASSIGN in 178 msec 2023-05-02 02:55:16,173 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682996116173"}]},"ts":"1682996116173"} 2023-05-02 02:55:16,174 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLED in hbase:meta 2023-05-02 02:55:16,176 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackForSplitTableRegionWithReplica to state=DISABLED 2023-05-02 02:55:16,178 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica in 195 msec 2023-05-02 02:55:16,286 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-05-02 02:55:16,286 INFO [Listener at localhost/37119] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 96 completed 2023-05-02 02:55:16,287 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testRollbackForSplitTableRegionWithReplica 2023-05-02 02:55:16,288 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure2.ProcedureExecutor(1029): Stored pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-02 02:55:16,289 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-02 02:55:16,290 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=101, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-02 02:55:16,290 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1227): Checking to see if procedure is done pid=101 2023-05-02 02:55:16,295 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/74b19512a655e6f00fc418d81fba16c8 2023-05-02 02:55:16,297 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/74b19512a655e6f00fc418d81fba16c8/cf1, FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/74b19512a655e6f00fc418d81fba16c8/recovered.edits] 2023-05-02 02:55:16,302 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/74b19512a655e6f00fc418d81fba16c8/recovered.edits/7.seqid to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRollbackForSplitTableRegionWithReplica/74b19512a655e6f00fc418d81fba16c8/recovered.edits/7.seqid 2023-05-02 02:55:16,303 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/74b19512a655e6f00fc418d81fba16c8 2023-05-02 02:55:16,303 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-05-02 02:55:16,305 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=101, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-02 02:55:16,307 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackForSplitTableRegionWithReplica from hbase:meta 2023-05-02 02:55:16,309 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackForSplitTableRegionWithReplica' descriptor. 2023-05-02 02:55:16,310 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=101, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-02 02:55:16,310 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackForSplitTableRegionWithReplica' from region states. 2023-05-02 02:55:16,310 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1682996050990.74b19512a655e6f00fc418d81fba16c8.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1682996116310"}]},"ts":"9223372036854775807"} 2023-05-02 02:55:16,310 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1682996050990_0001.1b91efe506b89bc57ba68287b021be4f.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1682996116310"}]},"ts":"9223372036854775807"} 2023-05-02 02:55:16,312 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 2 regions from META 2023-05-02 02:55:16,312 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 74b19512a655e6f00fc418d81fba16c8, NAME => 'testRollbackForSplitTableRegionWithReplica,,1682996050990.74b19512a655e6f00fc418d81fba16c8.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 1b91efe506b89bc57ba68287b021be4f, NAME => 'testRollbackForSplitTableRegionWithReplica,,1682996050990_0001.1b91efe506b89bc57ba68287b021be4f.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1}] 2023-05-02 02:55:16,312 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackForSplitTableRegionWithReplica' as deleted. 2023-05-02 02:55:16,312 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1682996116312"}]},"ts":"9223372036854775807"} 2023-05-02 02:55:16,313 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackForSplitTableRegionWithReplica state from META 2023-05-02 02:55:16,315 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=101, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-02 02:55:16,316 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica in 28 msec 2023-05-02 02:55:16,391 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1227): Checking to see if procedure is done pid=101 2023-05-02 02:55:16,392 INFO [Listener at localhost/37119] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 101 completed 2023-05-02 02:55:16,407 INFO [Listener at localhost/37119] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=348 (was 411), OpenFileDescriptor=661 (was 869), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=155 (was 254), ProcessCount=170 (was 170), AvailableMemoryMB=13822 (was 13836) 2023-05-02 02:55:16,419 INFO [Listener at localhost/37119] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=349, OpenFileDescriptor=661, MaxFileDescriptor=60000, SystemLoadAverage=155, ProcessCount=170, AvailableMemoryMB=13821 2023-05-02 02:55:16,419 WARN [Listener at localhost/37119] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-02 02:55:16,421 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-05-02 02:55:16,422 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-05-02 02:55:16,423 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(1029): Stored pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testInvalidSplitKey 2023-05-02 02:55:16,424 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_PRE_OPERATION 2023-05-02 02:55:16,425 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-05-02 02:55:16,427 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d 2023-05-02 02:55:16,427 DEBUG [HFileArchiver-10] backup.HFileArchiver(153): Directory hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d empty. 2023-05-02 02:55:16,427 DEBUG [HFileArchiver-10] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d 2023-05-02 02:55:16,427 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-05-02 02:55:16,436 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testInvalidSplitKey/.tabledesc/.tableinfo.0000000001 2023-05-02 02:55:16,437 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(7675): creating {ENCODED => 89e612e3d89dbdfa82cedca0a4b7689d, NAME => 'testInvalidSplitKey,,1682996116422.89e612e3d89dbdfa82cedca0a4b7689d.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testInvalidSplitKey', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp 2023-05-02 02:55:16,444 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(866): Instantiated testInvalidSplitKey,,1682996116422.89e612e3d89dbdfa82cedca0a4b7689d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-02 02:55:16,444 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1604): Closing 89e612e3d89dbdfa82cedca0a4b7689d, disabling compactions & flushes 2023-05-02 02:55:16,444 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1626): Closing region testInvalidSplitKey,,1682996116422.89e612e3d89dbdfa82cedca0a4b7689d. 2023-05-02 02:55:16,444 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testInvalidSplitKey,,1682996116422.89e612e3d89dbdfa82cedca0a4b7689d. 2023-05-02 02:55:16,444 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1714): Acquired close lock on testInvalidSplitKey,,1682996116422.89e612e3d89dbdfa82cedca0a4b7689d. after waiting 0 ms 2023-05-02 02:55:16,444 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1724): Updates disabled for region testInvalidSplitKey,,1682996116422.89e612e3d89dbdfa82cedca0a4b7689d. 2023-05-02 02:55:16,444 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1838): Closed testInvalidSplitKey,,1682996116422.89e612e3d89dbdfa82cedca0a4b7689d. 2023-05-02 02:55:16,444 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1558): Region close journal for 89e612e3d89dbdfa82cedca0a4b7689d: 2023-05-02 02:55:16,446 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ADD_TO_META 2023-05-02 02:55:16,446 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1682996116422.89e612e3d89dbdfa82cedca0a4b7689d.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1682996116446"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682996116446"}]},"ts":"1682996116446"} 2023-05-02 02:55:16,448 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-05-02 02:55:16,448 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-05-02 02:55:16,448 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682996116448"}]},"ts":"1682996116448"} 2023-05-02 02:55:16,449 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLING in hbase:meta 2023-05-02 02:55:16,453 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-05-02 02:55:16,454 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-05-02 02:55:16,454 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-05-02 02:55:16,454 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-05-02 02:55:16,454 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-05-02 02:55:16,454 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=89e612e3d89dbdfa82cedca0a4b7689d, ASSIGN}] 2023-05-02 02:55:16,455 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=89e612e3d89dbdfa82cedca0a4b7689d, ASSIGN 2023-05-02 02:55:16,456 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testInvalidSplitKey, region=89e612e3d89dbdfa82cedca0a4b7689d, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,38925,1682996021633; forceNewPlan=false, retain=false 2023-05-02 02:55:16,606 INFO [jenkins-hbase4:41119] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-05-02 02:55:16,607 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=89e612e3d89dbdfa82cedca0a4b7689d, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:16,607 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1682996116422.89e612e3d89dbdfa82cedca0a4b7689d.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1682996116607"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1682996116607"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682996116607"}]},"ts":"1682996116607"} 2023-05-02 02:55:16,609 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=104, ppid=103, state=RUNNABLE; OpenRegionProcedure 89e612e3d89dbdfa82cedca0a4b7689d, server=jenkins-hbase4.apache.org,38925,1682996021633}] 2023-05-02 02:55:16,765 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testInvalidSplitKey,,1682996116422.89e612e3d89dbdfa82cedca0a4b7689d. 2023-05-02 02:55:16,765 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 89e612e3d89dbdfa82cedca0a4b7689d, NAME => 'testInvalidSplitKey,,1682996116422.89e612e3d89dbdfa82cedca0a4b7689d.', STARTKEY => '', ENDKEY => ''} 2023-05-02 02:55:16,765 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-02 02:55:16,766 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testInvalidSplitKey 89e612e3d89dbdfa82cedca0a4b7689d 2023-05-02 02:55:16,766 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testInvalidSplitKey,,1682996116422.89e612e3d89dbdfa82cedca0a4b7689d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-02 02:55:16,766 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 89e612e3d89dbdfa82cedca0a4b7689d 2023-05-02 02:55:16,766 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 89e612e3d89dbdfa82cedca0a4b7689d 2023-05-02 02:55:16,767 INFO [StoreOpener-89e612e3d89dbdfa82cedca0a4b7689d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 89e612e3d89dbdfa82cedca0a4b7689d 2023-05-02 02:55:16,768 DEBUG [StoreOpener-89e612e3d89dbdfa82cedca0a4b7689d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/cf1 2023-05-02 02:55:16,768 DEBUG [StoreOpener-89e612e3d89dbdfa82cedca0a4b7689d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/cf1 2023-05-02 02:55:16,768 INFO [StoreOpener-89e612e3d89dbdfa82cedca0a4b7689d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 89e612e3d89dbdfa82cedca0a4b7689d columnFamilyName cf1 2023-05-02 02:55:16,769 INFO [StoreOpener-89e612e3d89dbdfa82cedca0a4b7689d-1] regionserver.HStore(310): Store=89e612e3d89dbdfa82cedca0a4b7689d/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-02 02:55:16,769 INFO [StoreOpener-89e612e3d89dbdfa82cedca0a4b7689d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 89e612e3d89dbdfa82cedca0a4b7689d 2023-05-02 02:55:16,770 DEBUG [StoreOpener-89e612e3d89dbdfa82cedca0a4b7689d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/cf2 2023-05-02 02:55:16,770 DEBUG [StoreOpener-89e612e3d89dbdfa82cedca0a4b7689d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/cf2 2023-05-02 02:55:16,770 INFO [StoreOpener-89e612e3d89dbdfa82cedca0a4b7689d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 89e612e3d89dbdfa82cedca0a4b7689d columnFamilyName cf2 2023-05-02 02:55:16,771 INFO [StoreOpener-89e612e3d89dbdfa82cedca0a4b7689d-1] regionserver.HStore(310): Store=89e612e3d89dbdfa82cedca0a4b7689d/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-02 02:55:16,771 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d 2023-05-02 02:55:16,772 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d 2023-05-02 02:55:16,773 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testInvalidSplitKey descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-02 02:55:16,774 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 89e612e3d89dbdfa82cedca0a4b7689d 2023-05-02 02:55:16,776 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-05-02 02:55:16,776 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 89e612e3d89dbdfa82cedca0a4b7689d; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70464386, jitterRate=0.050001174211502075}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-02 02:55:16,776 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 89e612e3d89dbdfa82cedca0a4b7689d: 2023-05-02 02:55:16,777 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testInvalidSplitKey,,1682996116422.89e612e3d89dbdfa82cedca0a4b7689d., pid=104, masterSystemTime=1682996116761 2023-05-02 02:55:16,778 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testInvalidSplitKey,,1682996116422.89e612e3d89dbdfa82cedca0a4b7689d. 2023-05-02 02:55:16,778 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testInvalidSplitKey,,1682996116422.89e612e3d89dbdfa82cedca0a4b7689d. 2023-05-02 02:55:16,779 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=89e612e3d89dbdfa82cedca0a4b7689d, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:16,779 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testInvalidSplitKey,,1682996116422.89e612e3d89dbdfa82cedca0a4b7689d.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1682996116779"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1682996116779"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1682996116779"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682996116779"}]},"ts":"1682996116779"} 2023-05-02 02:55:16,782 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=104, resume processing ppid=103 2023-05-02 02:55:16,782 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=104, ppid=103, state=SUCCESS; OpenRegionProcedure 89e612e3d89dbdfa82cedca0a4b7689d, server=jenkins-hbase4.apache.org,38925,1682996021633 in 171 msec 2023-05-02 02:55:16,784 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=103, resume processing ppid=102 2023-05-02 02:55:16,784 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=103, ppid=102, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=89e612e3d89dbdfa82cedca0a4b7689d, ASSIGN in 328 msec 2023-05-02 02:55:16,785 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-05-02 02:55:16,785 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682996116785"}]},"ts":"1682996116785"} 2023-05-02 02:55:16,786 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLED in hbase:meta 2023-05-02 02:55:16,788 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_POST_OPERATION 2023-05-02 02:55:16,789 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey in 366 msec 2023-05-02 02:55:16,931 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-02 02:55:16,933 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-05-02 02:55:16,933 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-05-02 02:55:16,933 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:16,933 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-05-02 02:55:16,933 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-05-02 02:55:16,933 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:16,934 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-02 02:55:16,935 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:16,935 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:16,935 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:16,935 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:16,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:16,935 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:16,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:16,935 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:16,935 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:16,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:16,935 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:16,935 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-02 02:55:16,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-02 02:55:16,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-02 02:55:16,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-02 02:55:16,935 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:16,936 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:16,936 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:16,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:16,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-02 02:55:16,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:16,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-02 02:55:16,936 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-02 02:55:16,936 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-02 02:55:16,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:16,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-02 02:55:16,936 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-02 02:55:16,936 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-02 02:55:16,936 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-02 02:55:16,936 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-02 02:55:16,936 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:16,936 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-02 02:55:16,937 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:16,937 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-02 02:55:16,937 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-02 02:55:16,937 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-02 02:55:16,937 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42019,1682996021676' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-02 02:55:16,937 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-02 02:55:16,937 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-02 02:55:16,937 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-02 02:55:16,937 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1682996116422.89e612e3d89dbdfa82cedca0a4b7689d. 2023-05-02 02:55:16,938 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:16,937 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-02 02:55:16,938 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-02 02:55:16,938 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-02 02:55:16,938 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1682996116422.89e612e3d89dbdfa82cedca0a4b7689d. started... 2023-05-02 02:55:16,938 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-02 02:55:16,939 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 89e612e3d89dbdfa82cedca0a4b7689d 2/2 column families, dataSize=120 B heapSize=928 B 2023-05-02 02:55:16,939 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:16,940 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:16,940 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:16,940 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:16,940 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:16,940 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:16,950 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/.tmp/cf1/113e44b50dc14014b93fbc92b1c1025e 2023-05-02 02:55:16,961 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/.tmp/cf2/4a4d0a972e7c4a988ba56eece569e63c 2023-05-02 02:55:16,965 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/.tmp/cf1/113e44b50dc14014b93fbc92b1c1025e as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/cf1/113e44b50dc14014b93fbc92b1c1025e 2023-05-02 02:55:16,971 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/cf1/113e44b50dc14014b93fbc92b1c1025e, entries=2, sequenceid=6, filesize=4.8 K 2023-05-02 02:55:16,972 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/.tmp/cf2/4a4d0a972e7c4a988ba56eece569e63c as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/cf2/4a4d0a972e7c4a988ba56eece569e63c 2023-05-02 02:55:16,975 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/cf2/4a4d0a972e7c4a988ba56eece569e63c, entries=2, sequenceid=6, filesize=4.8 K 2023-05-02 02:55:16,976 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 89e612e3d89dbdfa82cedca0a4b7689d in 37ms, sequenceid=6, compaction requested=false 2023-05-02 02:55:16,976 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testInvalidSplitKey' 2023-05-02 02:55:16,976 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 89e612e3d89dbdfa82cedca0a4b7689d: 2023-05-02 02:55:16,977 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1682996116422.89e612e3d89dbdfa82cedca0a4b7689d. 2023-05-02 02:55:16,977 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-02 02:55:16,977 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-02 02:55:16,977 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:16,977 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-02 02:55:16,977 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38925,1682996021633' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-02 02:55:16,978 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:16,978 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:16,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:16,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:16,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:16,978 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:16,979 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:16,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:16,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:16,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-02 02:55:16,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:16,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:16,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:16,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:16,980 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38925,1682996021633' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-05-02 02:55:16,980 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2a9f6a01[Count = 0] remaining members to acquire global barrier 2023-05-02 02:55:16,980 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-05-02 02:55:16,980 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:16,981 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:16,981 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:16,981 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:16,982 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:16,981 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:16,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:16,982 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:16,982 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-02 02:55:16,982 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-02 02:55:16,982 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,39081,1682996021579' in zk 2023-05-02 02:55:16,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:16,982 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-02 02:55:16,982 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-02 02:55:16,982 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-02 02:55:16,982 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,38925,1682996021633' in zk 2023-05-02 02:55:16,982 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:16,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:16,982 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-02 02:55:16,982 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-02 02:55:16,982 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,42019,1682996021676' in zk 2023-05-02 02:55:16,984 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-02 02:55:16,984 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:16,984 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-02 02:55:16,985 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:16,985 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-02 02:55:16,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:16,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:16,985 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:16,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:16,985 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-02 02:55:16,985 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-02 02:55:16,985 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:16,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:16,985 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-02 02:55:16,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:16,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-02 02:55:16,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:16,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:16,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:16,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:16,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-02 02:55:16,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:16,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:16,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:16,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,38925,1682996021633': 2023-05-02 02:55:16,988 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38925,1682996021633' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-05-02 02:55:16,988 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-05-02 02:55:16,988 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-02 02:55:16,988 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-02 02:55:16,988 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-05-02 02:55:16,988 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-02 02:55:16,990 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:16,990 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:16,990 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:16,990 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:16,990 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:16,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:16,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:16,990 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:16,990 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:16,990 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:16,990 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:16,990 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:16,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:16,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:16,990 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:16,990 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:16,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:16,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:16,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:16,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:16,991 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:16,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:16,991 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:16,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:16,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:16,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:16,991 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:16,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:16,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-02 02:55:16,991 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:16,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:16,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-02 02:55:16,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:16,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:16,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:16,993 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:16,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:16,993 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:16,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-02 02:55:16,993 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:16,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:16,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:16,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:16,997 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:16,997 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:16,997 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:16,997 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:16,997 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:16,997 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:16,997 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:16,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:16,997 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:16,997 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-02 02:55:16,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:16,997 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-05-02 02:55:16,997 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-02 02:55:16,997 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:16,997 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:16,997 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:16,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:16,997 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:16,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:16,997 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-05-02 02:55:16,997 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:16,998 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-02 02:55:16,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:16,997 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:16,998 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:16,998 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-02 02:55:16,998 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:16,998 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:16,998 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:16,998 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:16,998 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:16,998 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:16,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:16,998 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,098 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-02 02:55:17,099 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-02 02:55:17,110 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-02 02:55:17,112 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-05-02 02:55:17,112 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-05-02 02:55:17,112 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-05-02 02:55:17,112 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:17,112 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-05-02 02:55:17,112 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-05-02 02:55:17,113 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,113 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-02 02:55:17,114 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:17,114 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:17,114 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:17,114 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:17,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:17,114 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:17,114 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:17,114 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:17,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:17,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-02 02:55:17,114 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-02 02:55:17,114 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-02 02:55:17,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-02 02:55:17,114 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,115 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,115 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:17,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-02 02:55:17,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:17,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-02 02:55:17,115 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-02 02:55:17,115 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-02 02:55:17,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:17,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-02 02:55:17,115 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-02 02:55:17,115 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-05-02 02:55:17,115 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-02 02:55:17,115 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-02 02:55:17,115 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:17,115 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-02 02:55:17,116 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-02 02:55:17,116 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42019,1682996021676' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-02 02:55:17,116 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-05-02 02:55:17,116 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-02 02:55:17,116 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-02 02:55:17,116 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-05-02 02:55:17,116 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:17,116 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-02 02:55:17,116 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-02 02:55:17,116 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-02 02:55:17,117 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:17,116 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-02 02:55:17,117 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-02 02:55:17,117 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-02 02:55:17,117 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-02 02:55:17,117 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-02 02:55:17,117 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1682996116422.89e612e3d89dbdfa82cedca0a4b7689d. 2023-05-02 02:55:17,117 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,117 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1682996116422.89e612e3d89dbdfa82cedca0a4b7689d. started... 2023-05-02 02:55:17,117 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 89e612e3d89dbdfa82cedca0a4b7689d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-02 02:55:17,126 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,126 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,126 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:17,126 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,126 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:17,127 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/.tmp/cf1/4a54f18026f2416ba9e253e5e5888c9e 2023-05-02 02:55:17,139 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/.tmp/cf2/534e0bc86fa24bd18c3a1abcede64e32 2023-05-02 02:55:17,143 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/.tmp/cf1/4a54f18026f2416ba9e253e5e5888c9e as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/cf1/4a54f18026f2416ba9e253e5e5888c9e 2023-05-02 02:55:17,148 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/cf1/4a54f18026f2416ba9e253e5e5888c9e, entries=10, sequenceid=19, filesize=5.0 K 2023-05-02 02:55:17,149 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/.tmp/cf2/534e0bc86fa24bd18c3a1abcede64e32 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/cf2/534e0bc86fa24bd18c3a1abcede64e32 2023-05-02 02:55:17,153 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/cf2/534e0bc86fa24bd18c3a1abcede64e32, entries=10, sequenceid=19, filesize=5.0 K 2023-05-02 02:55:17,154 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 89e612e3d89dbdfa82cedca0a4b7689d in 37ms, sequenceid=19, compaction requested=false 2023-05-02 02:55:17,154 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 89e612e3d89dbdfa82cedca0a4b7689d: 2023-05-02 02:55:17,154 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1682996116422.89e612e3d89dbdfa82cedca0a4b7689d. 2023-05-02 02:55:17,154 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-02 02:55:17,154 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-02 02:55:17,154 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,155 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-02 02:55:17,155 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38925,1682996021633' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-02 02:55:17,156 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,156 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:17,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:17,156 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,156 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:17,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:17,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:17,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-02 02:55:17,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:17,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:17,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:17,158 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38925,1682996021633' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-05-02 02:55:17,158 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5a1fa642[Count = 0] remaining members to acquire global barrier 2023-05-02 02:55:17,158 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-05-02 02:55:17,159 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,160 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,160 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,160 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,160 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,160 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-02 02:55:17,160 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-02 02:55:17,160 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,160 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,42019,1682996021676' in zk 2023-05-02 02:55:17,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,160 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,160 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-02 02:55:17,160 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-02 02:55:17,160 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-02 02:55:17,160 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,160 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,38925,1682996021633' in zk 2023-05-02 02:55:17,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,160 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-02 02:55:17,161 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-02 02:55:17,161 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,39081,1682996021579' in zk 2023-05-02 02:55:17,161 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-02 02:55:17,161 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:17,161 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-02 02:55:17,163 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-02 02:55:17,163 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:17,163 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-02 02:55:17,164 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-02 02:55:17,164 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:17,164 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,164 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-02 02:55:17,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:17,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:17,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:17,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:17,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-02 02:55:17,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:17,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:17,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:17,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-02 02:55:17,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:17,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:17,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,38925,1682996021633': 2023-05-02 02:55:17,167 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38925,1682996021633' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-05-02 02:55:17,167 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-05-02 02:55:17,167 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-02 02:55:17,167 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-02 02:55:17,167 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-05-02 02:55:17,167 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-02 02:55:17,169 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,169 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,169 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,169 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:17,169 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:17,169 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,169 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,169 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,169 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:17,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:17,169 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:17,170 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:17,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:17,170 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:17,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:17,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,170 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:17,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:17,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:17,171 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:17,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-02 02:55:17,171 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:17,171 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:17,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-02 02:55:17,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:17,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:17,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:17,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-02 02:55:17,173 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:17,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:17,174 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,174 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:17,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:17,176 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:17,176 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:17,176 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:17,176 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:17,176 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:17,176 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:17,176 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:17,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:17,176 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:17,177 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-05-02 02:55:17,177 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-02 02:55:17,176 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:17,176 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:17,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:17,177 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:17,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:17,176 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-02 02:55:17,177 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-05-02 02:55:17,177 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:17,177 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:17,177 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:17,177 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-02 02:55:17,177 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:17,177 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,177 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,177 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:17,177 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,178 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,177 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-02 02:55:17,177 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:17,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:17,178 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:17,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:17,278 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-02 02:55:17,279 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-02 02:55:17,291 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-02 02:55:17,293 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-05-02 02:55:17,293 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-05-02 02:55:17,293 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-05-02 02:55:17,293 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:17,293 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-05-02 02:55:17,293 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-05-02 02:55:17,293 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,294 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-02 02:55:17,296 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,296 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:17,296 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:17,296 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:17,296 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:17,296 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:17,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:17,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:17,296 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,296 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-02 02:55:17,296 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:17,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:17,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-02 02:55:17,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-02 02:55:17,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-02 02:55:17,297 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,297 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,297 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:17,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-02 02:55:17,297 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-02 02:55:17,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:17,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:17,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-02 02:55:17,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-02 02:55:17,297 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-02 02:55:17,297 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-02 02:55:17,297 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-05-02 02:55:17,297 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-02 02:55:17,297 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-02 02:55:17,297 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:17,297 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-05-02 02:55:17,298 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-05-02 02:55:17,297 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-02 02:55:17,298 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-02 02:55:17,298 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-02 02:55:17,298 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-02 02:55:17,298 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-02 02:55:17,298 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:17,298 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-02 02:55:17,298 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-02 02:55:17,298 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-02 02:55:17,298 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:17,298 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-02 02:55:17,298 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-02 02:55:17,298 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1682996116422.89e612e3d89dbdfa82cedca0a4b7689d. 2023-05-02 02:55:17,298 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-02 02:55:17,298 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-02 02:55:17,298 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1682996116422.89e612e3d89dbdfa82cedca0a4b7689d. started... 2023-05-02 02:55:17,298 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42019,1682996021676' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-02 02:55:17,298 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 89e612e3d89dbdfa82cedca0a4b7689d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-02 02:55:17,299 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,300 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,300 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,300 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:17,300 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,300 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:17,310 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/.tmp/cf1/5b4a3bd9b34b4c9ca2aef0e697c63661 2023-05-02 02:55:17,322 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/.tmp/cf2/60270d6a1c7f4ab18f57039b883c50d5 2023-05-02 02:55:17,326 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/.tmp/cf1/5b4a3bd9b34b4c9ca2aef0e697c63661 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/cf1/5b4a3bd9b34b4c9ca2aef0e697c63661 2023-05-02 02:55:17,329 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/cf1/5b4a3bd9b34b4c9ca2aef0e697c63661, entries=10, sequenceid=32, filesize=5.0 K 2023-05-02 02:55:17,330 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/.tmp/cf2/60270d6a1c7f4ab18f57039b883c50d5 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/cf2/60270d6a1c7f4ab18f57039b883c50d5 2023-05-02 02:55:17,334 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/cf2/60270d6a1c7f4ab18f57039b883c50d5, entries=10, sequenceid=32, filesize=5.0 K 2023-05-02 02:55:17,335 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 89e612e3d89dbdfa82cedca0a4b7689d in 37ms, sequenceid=32, compaction requested=true 2023-05-02 02:55:17,335 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 89e612e3d89dbdfa82cedca0a4b7689d: 2023-05-02 02:55:17,335 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1682996116422.89e612e3d89dbdfa82cedca0a4b7689d. 2023-05-02 02:55:17,335 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-02 02:55:17,335 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-02 02:55:17,335 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,335 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-02 02:55:17,335 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38925,1682996021633' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-02 02:55:17,337 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,337 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:17,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:17,337 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,337 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:17,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:17,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:17,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-02 02:55:17,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:17,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:17,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:17,339 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38925,1682996021633' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-05-02 02:55:17,339 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6ba2829e[Count = 0] remaining members to acquire global barrier 2023-05-02 02:55:17,339 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-05-02 02:55:17,339 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,341 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,341 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,341 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,341 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,341 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,341 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-02 02:55:17,341 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,341 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-02 02:55:17,342 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-02 02:55:17,341 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,342 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,39081,1682996021579' in zk 2023-05-02 02:55:17,342 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-02 02:55:17,342 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-02 02:55:17,342 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-02 02:55:17,342 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,38925,1682996021633' in zk 2023-05-02 02:55:17,342 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-02 02:55:17,342 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,42019,1682996021676' in zk 2023-05-02 02:55:17,343 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-02 02:55:17,343 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:17,343 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-02 02:55:17,344 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-02 02:55:17,344 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:17,344 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-02 02:55:17,344 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:17,344 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,344 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-02 02:55:17,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:17,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:17,344 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-02 02:55:17,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:17,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:17,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-02 02:55:17,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:17,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:17,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:17,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-02 02:55:17,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:17,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:17,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,38925,1682996021633': 2023-05-02 02:55:17,347 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38925,1682996021633' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-05-02 02:55:17,347 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-05-02 02:55:17,347 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-02 02:55:17,347 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-02 02:55:17,347 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-05-02 02:55:17,347 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-02 02:55:17,349 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,349 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,349 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,349 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:17,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:17,349 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:17,349 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,349 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,349 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:17,349 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,349 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:17,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:17,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,349 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:17,350 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:17,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:17,350 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:17,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:17,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-02 02:55:17,350 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:17,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:17,350 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:17,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,353 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:17,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-02 02:55:17,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:17,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:17,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:17,354 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-02 02:55:17,355 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:17,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,355 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:17,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:17,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:17,361 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:17,361 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:17,361 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:17,361 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:17,361 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-05-02 02:55:17,362 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-02 02:55:17,361 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:17,361 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:17,361 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:17,361 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:17,361 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:17,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:17,361 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-02 02:55:17,362 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-05-02 02:55:17,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:17,362 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:17,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:17,362 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:17,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:17,362 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:17,362 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-02 02:55:17,362 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,362 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:17,362 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:17,362 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:17,362 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-02 02:55:17,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:17,363 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:17,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:17,363 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,363 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:17,363 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,363 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,363 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,462 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-02 02:55:17,463 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-02 02:55:17,476 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-02 02:55:17,478 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-05-02 02:55:17,478 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-05-02 02:55:17,478 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-05-02 02:55:17,478 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:17,478 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-05-02 02:55:17,478 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-05-02 02:55:17,479 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,479 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-02 02:55:17,480 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:17,480 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:17,480 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:17,480 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:17,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:17,480 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:17,480 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:17,480 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:17,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:17,484 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-02 02:55:17,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-02 02:55:17,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-02 02:55:17,484 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-02 02:55:17,484 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,484 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,484 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:17,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-02 02:55:17,485 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-02 02:55:17,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:17,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-02 02:55:17,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:17,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-02 02:55:17,485 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-05-02 02:55:17,485 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-02 02:55:17,485 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-02 02:55:17,485 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-02 02:55:17,485 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-02 02:55:17,485 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:17,485 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-02 02:55:17,485 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-05-02 02:55:17,485 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-02 02:55:17,485 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-02 02:55:17,485 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-05-02 02:55:17,485 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-02 02:55:17,486 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-02 02:55:17,485 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-02 02:55:17,486 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:17,486 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-02 02:55:17,486 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-02 02:55:17,486 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-02 02:55:17,486 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:17,486 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-02 02:55:17,486 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42019,1682996021676' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-02 02:55:17,486 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-02 02:55:17,486 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-02 02:55:17,486 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1682996116422.89e612e3d89dbdfa82cedca0a4b7689d. 2023-05-02 02:55:17,486 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1682996116422.89e612e3d89dbdfa82cedca0a4b7689d. started... 2023-05-02 02:55:17,486 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 89e612e3d89dbdfa82cedca0a4b7689d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-02 02:55:17,487 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,488 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,488 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,488 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:17,488 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,488 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:17,499 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/.tmp/cf1/34100ad5828540969babaa2f2ea76045 2023-05-02 02:55:17,512 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/.tmp/cf2/1dc04889d28f49bb9a649d4a31083f7c 2023-05-02 02:55:17,516 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/.tmp/cf1/34100ad5828540969babaa2f2ea76045 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/cf1/34100ad5828540969babaa2f2ea76045 2023-05-02 02:55:17,520 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/cf1/34100ad5828540969babaa2f2ea76045, entries=10, sequenceid=45, filesize=5.0 K 2023-05-02 02:55:17,521 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/.tmp/cf2/1dc04889d28f49bb9a649d4a31083f7c as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/cf2/1dc04889d28f49bb9a649d4a31083f7c 2023-05-02 02:55:17,525 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/cf2/1dc04889d28f49bb9a649d4a31083f7c, entries=10, sequenceid=45, filesize=5.0 K 2023-05-02 02:55:17,525 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 89e612e3d89dbdfa82cedca0a4b7689d in 39ms, sequenceid=45, compaction requested=true 2023-05-02 02:55:17,525 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 89e612e3d89dbdfa82cedca0a4b7689d: 2023-05-02 02:55:17,526 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1682996116422.89e612e3d89dbdfa82cedca0a4b7689d. 2023-05-02 02:55:17,526 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-02 02:55:17,526 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-02 02:55:17,526 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,526 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-02 02:55:17,526 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38925,1682996021633' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-02 02:55:17,529 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,529 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:17,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:17,529 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,529 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:17,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:17,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:17,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-02 02:55:17,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:17,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:17,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:17,531 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38925,1682996021633' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-05-02 02:55:17,531 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@481926ce[Count = 0] remaining members to acquire global barrier 2023-05-02 02:55:17,531 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-05-02 02:55:17,531 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,532 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,532 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,532 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,532 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,532 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-02 02:55:17,532 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-02 02:55:17,532 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,38925,1682996021633' in zk 2023-05-02 02:55:17,532 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,532 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-02 02:55:17,532 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,532 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-02 02:55:17,532 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,532 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-02 02:55:17,533 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,39081,1682996021579' in zk 2023-05-02 02:55:17,533 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-02 02:55:17,533 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-02 02:55:17,533 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,42019,1682996021676' in zk 2023-05-02 02:55:17,533 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:17,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:17,533 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-02 02:55:17,534 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:17,534 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-02 02:55:17,534 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-02 02:55:17,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:17,534 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-02 02:55:17,534 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:17,534 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:17,535 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-02 02:55:17,535 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-02 02:55:17,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:17,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-02 02:55:17,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:17,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:17,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:17,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-02 02:55:17,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:17,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:17,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,38925,1682996021633': 2023-05-02 02:55:17,537 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38925,1682996021633' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-05-02 02:55:17,537 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-05-02 02:55:17,538 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-02 02:55:17,538 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-02 02:55:17,538 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-05-02 02:55:17,538 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-02 02:55:17,545 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,545 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,545 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:17,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:17,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:17,545 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,545 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,545 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,545 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:17,545 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,545 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:17,545 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:17,545 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:17,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,545 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,546 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:17,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:17,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-02 02:55:17,546 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:17,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:17,546 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:17,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,546 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:17,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:17,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:17,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-02 02:55:17,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:17,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:17,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:17,547 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-02 02:55:17,548 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:17,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,548 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:17,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:17,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:17,551 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:17,551 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-02 02:55:17,551 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,551 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:17,551 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:17,551 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:17,551 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-05-02 02:55:17,551 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-02 02:55:17,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:17,551 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:17,551 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:17,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:17,552 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-05-02 02:55:17,552 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-02 02:55:17,551 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:17,551 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:17,551 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:17,551 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:17,552 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:17,552 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-02 02:55:17,552 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:17,552 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:17,552 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:17,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:17,552 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:17,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:17,552 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:17,552 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:17,552 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,552 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,552 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,553 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:17,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:17,652 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-02 02:55:17,653 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-02 02:55:17,663 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-02 02:55:17,665 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-05-02 02:55:17,665 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-05-02 02:55:17,665 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-05-02 02:55:17,665 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:17,665 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-05-02 02:55:17,665 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-05-02 02:55:17,665 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,665 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-02 02:55:17,666 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:17,666 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:17,666 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:17,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:17,666 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:17,666 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,666 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:17,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:17,667 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:17,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:17,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-02 02:55:17,667 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,667 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-02 02:55:17,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-02 02:55:17,667 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-02 02:55:17,667 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:17,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-02 02:55:17,667 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,667 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-02 02:55:17,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:17,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-02 02:55:17,668 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-02 02:55:17,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:17,668 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-05-02 02:55:17,668 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-05-02 02:55:17,668 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-02 02:55:17,668 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-02 02:55:17,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-02 02:55:17,668 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-02 02:55:17,668 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-02 02:55:17,668 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-02 02:55:17,668 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:17,668 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:17,668 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-02 02:55:17,668 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-02 02:55:17,668 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-05-02 02:55:17,668 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42019,1682996021676' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-02 02:55:17,668 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-02 02:55:17,668 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-02 02:55:17,669 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-02 02:55:17,669 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-02 02:55:17,669 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-02 02:55:17,669 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1682996116422.89e612e3d89dbdfa82cedca0a4b7689d. 2023-05-02 02:55:17,669 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:17,669 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1682996116422.89e612e3d89dbdfa82cedca0a4b7689d. started... 2023-05-02 02:55:17,669 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-02 02:55:17,669 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-02 02:55:17,669 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-02 02:55:17,669 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 89e612e3d89dbdfa82cedca0a4b7689d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-02 02:55:17,670 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,672 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,672 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:17,672 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,672 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,672 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:17,677 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/.tmp/cf1/55644b1330dc496aa5f3690bfbaa5507 2023-05-02 02:55:17,688 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/.tmp/cf2/aa37ef94f30546379939765b0a5e0515 2023-05-02 02:55:17,692 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/.tmp/cf1/55644b1330dc496aa5f3690bfbaa5507 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/cf1/55644b1330dc496aa5f3690bfbaa5507 2023-05-02 02:55:17,696 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/cf1/55644b1330dc496aa5f3690bfbaa5507, entries=10, sequenceid=58, filesize=5.0 K 2023-05-02 02:55:17,697 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/.tmp/cf2/aa37ef94f30546379939765b0a5e0515 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/cf2/aa37ef94f30546379939765b0a5e0515 2023-05-02 02:55:17,701 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/cf2/aa37ef94f30546379939765b0a5e0515, entries=10, sequenceid=58, filesize=5.0 K 2023-05-02 02:55:17,702 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 89e612e3d89dbdfa82cedca0a4b7689d in 33ms, sequenceid=58, compaction requested=true 2023-05-02 02:55:17,702 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 89e612e3d89dbdfa82cedca0a4b7689d: 2023-05-02 02:55:17,702 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1682996116422.89e612e3d89dbdfa82cedca0a4b7689d. 2023-05-02 02:55:17,702 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-02 02:55:17,702 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-02 02:55:17,702 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,703 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-02 02:55:17,703 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38925,1682996021633' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-02 02:55:17,704 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,704 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:17,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:17,704 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,704 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:17,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:17,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:17,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-02 02:55:17,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:17,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:17,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:17,706 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38925,1682996021633' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-05-02 02:55:17,706 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@190782ae[Count = 0] remaining members to acquire global barrier 2023-05-02 02:55:17,706 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-05-02 02:55:17,706 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,707 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,707 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,707 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,707 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,707 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,707 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,707 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-02 02:55:17,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,707 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-02 02:55:17,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,707 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,39081,1682996021579' in zk 2023-05-02 02:55:17,707 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-02 02:55:17,707 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-02 02:55:17,707 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,42019,1682996021676' in zk 2023-05-02 02:55:17,707 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,707 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-02 02:55:17,707 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-02 02:55:17,708 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-02 02:55:17,708 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,38925,1682996021633' in zk 2023-05-02 02:55:17,712 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-02 02:55:17,712 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-02 02:55:17,712 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:17,712 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:17,712 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-02 02:55:17,712 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-02 02:55:17,715 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-02 02:55:17,715 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,715 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:17,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,715 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-02 02:55:17,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:17,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:17,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:17,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:17,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-02 02:55:17,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:17,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:17,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:17,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-02 02:55:17,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:17,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:17,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,38925,1682996021633': 2023-05-02 02:55:17,718 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38925,1682996021633' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-05-02 02:55:17,718 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-05-02 02:55:17,718 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-02 02:55:17,718 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-02 02:55:17,718 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-05-02 02:55:17,718 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-02 02:55:17,720 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,720 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,720 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,720 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:17,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:17,720 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,720 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:17,720 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,720 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:17,720 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,720 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:17,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:17,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,720 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:17,720 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:17,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:17,721 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:17,721 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:17,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:17,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-02 02:55:17,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:17,721 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:17,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,721 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:17,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-02 02:55:17,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:17,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:17,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:17,723 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:17,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-02 02:55:17,723 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:17,723 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:17,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:17,727 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:17,727 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:17,727 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-02 02:55:17,727 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:17,727 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,727 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:17,727 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:17,727 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:17,727 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:17,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:17,727 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-05-02 02:55:17,727 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-02 02:55:17,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:17,727 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:17,727 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:17,728 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-02 02:55:17,728 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:17,728 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,728 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,728 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:17,728 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,728 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,728 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-05-02 02:55:17,728 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-02 02:55:17,728 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:17,728 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:17,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:17,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:17,732 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:17,732 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:17,732 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:17,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:17,732 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:17,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:17,828 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-02 02:55:17,829 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-02 02:55:17,840 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-02 02:55:17,841 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-05-02 02:55:17,842 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-05-02 02:55:17,842 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-05-02 02:55:17,842 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:17,842 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-05-02 02:55:17,842 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-05-02 02:55:17,842 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,842 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-02 02:55:17,843 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:17,843 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,843 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:17,843 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:17,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:17,843 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:17,844 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:17,844 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:17,844 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,844 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-02 02:55:17,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:17,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:17,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-02 02:55:17,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-02 02:55:17,844 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-02 02:55:17,844 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:17,845 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-02 02:55:17,845 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-02 02:55:17,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:17,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-02 02:55:17,845 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-02 02:55:17,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:17,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-02 02:55:17,845 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-02 02:55:17,845 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-05-02 02:55:17,845 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-02 02:55:17,845 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-05-02 02:55:17,845 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-02 02:55:17,845 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-05-02 02:55:17,845 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-02 02:55:17,845 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-02 02:55:17,845 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-02 02:55:17,845 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-02 02:55:17,845 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:17,845 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:17,845 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-02 02:55:17,845 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-02 02:55:17,846 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-02 02:55:17,845 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:17,846 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42019,1682996021676' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-02 02:55:17,846 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-02 02:55:17,846 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-02 02:55:17,846 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-02 02:55:17,846 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1682996116422.89e612e3d89dbdfa82cedca0a4b7689d. 2023-05-02 02:55:17,846 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-02 02:55:17,846 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-02 02:55:17,846 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1682996116422.89e612e3d89dbdfa82cedca0a4b7689d. started... 2023-05-02 02:55:17,846 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 89e612e3d89dbdfa82cedca0a4b7689d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-02 02:55:17,847 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,848 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,848 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,848 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:17,848 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,848 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:17,856 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/.tmp/cf1/a05aeeaa8ac5441a8472176dc94d2342 2023-05-02 02:55:17,869 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/.tmp/cf2/175e42f3827d46ef89219b73f0e456f2 2023-05-02 02:55:17,872 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/.tmp/cf1/a05aeeaa8ac5441a8472176dc94d2342 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/cf1/a05aeeaa8ac5441a8472176dc94d2342 2023-05-02 02:55:17,876 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/cf1/a05aeeaa8ac5441a8472176dc94d2342, entries=10, sequenceid=71, filesize=5.0 K 2023-05-02 02:55:17,877 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/.tmp/cf2/175e42f3827d46ef89219b73f0e456f2 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/cf2/175e42f3827d46ef89219b73f0e456f2 2023-05-02 02:55:17,881 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/cf2/175e42f3827d46ef89219b73f0e456f2, entries=10, sequenceid=71, filesize=5.0 K 2023-05-02 02:55:17,882 INFO [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 89e612e3d89dbdfa82cedca0a4b7689d in 35ms, sequenceid=71, compaction requested=true 2023-05-02 02:55:17,882 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 89e612e3d89dbdfa82cedca0a4b7689d: 2023-05-02 02:55:17,882 DEBUG [rs(jenkins-hbase4.apache.org,38925,1682996021633)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1682996116422.89e612e3d89dbdfa82cedca0a4b7689d. 2023-05-02 02:55:17,882 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-02 02:55:17,882 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-02 02:55:17,882 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,882 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-02 02:55:17,882 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38925,1682996021633' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-02 02:55:17,884 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,884 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:17,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:17,885 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,885 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:17,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:17,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:17,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-02 02:55:17,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:17,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:17,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:17,887 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38925,1682996021633' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-05-02 02:55:17,887 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3e0d0a7a[Count = 0] remaining members to acquire global barrier 2023-05-02 02:55:17,887 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-05-02 02:55:17,887 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,888 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,888 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,888 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,888 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,888 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,888 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,889 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-02 02:55:17,889 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-02 02:55:17,889 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,42019,1682996021676' in zk 2023-05-02 02:55:17,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,889 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,889 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-02 02:55:17,889 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-02 02:55:17,889 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-02 02:55:17,889 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-02 02:55:17,889 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-02 02:55:17,889 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,38925,1682996021633' in zk 2023-05-02 02:55:17,889 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,39081,1682996021579' in zk 2023-05-02 02:55:17,890 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-02 02:55:17,890 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:17,890 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-02 02:55:17,891 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-02 02:55:17,891 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:17,891 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,891 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-02 02:55:17,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:17,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:17,891 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-02 02:55:17,891 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:17,891 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-02 02:55:17,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:17,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:17,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-02 02:55:17,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:17,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:17,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:17,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-02 02:55:17,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:17,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:17,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,38925,1682996021633': 2023-05-02 02:55:17,894 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38925,1682996021633' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-05-02 02:55:17,894 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-05-02 02:55:17,894 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-02 02:55:17,894 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-02 02:55:17,894 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-05-02 02:55:17,894 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-02 02:55:17,897 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,897 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,897 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:17,897 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:17,897 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,897 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:17,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:17,897 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,897 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:17,897 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,897 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:17,898 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:17,898 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:17,898 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:17,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:17,898 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:17,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:17,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:17,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-02 02:55:17,898 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:17,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:17,899 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:17,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-02 02:55:17,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:17,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:17,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:17,900 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:17,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-02 02:55:17,901 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:17,901 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:17,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:17,904 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:17,904 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:17,904 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:17,904 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:17,904 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:17,904 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:17,904 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:17,904 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:17,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:17,904 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:17,904 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:17,904 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-05-02 02:55:17,904 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-02 02:55:17,904 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:17,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:17,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:17,904 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:17,904 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-02 02:55:17,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:17,905 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,905 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-05-02 02:55:17,905 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:17,905 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:17,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:17,905 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-02 02:55:17,905 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:17,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:17,905 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-02 02:55:17,905 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:17,905 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,905 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:17,905 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:17,905 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-02 02:55:17,905 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-02 02:55:18,005 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-02 02:55:18,006 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-02 02:55:18,014 INFO [Listener at localhost/37119] assignment.SplitTableRegionProcedure(206): testInvalidSplitKey,,1682996116422.89e612e3d89dbdfa82cedca0a4b7689d. splitKey isn't explicitly specified, will try to find a best split key from RS jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:18,015 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38925] regionserver.StoreUtils(129): cannot split hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/cf1/4a54f18026f2416ba9e253e5e5888c9e because midkey is the same as first or last row 2023-05-02 02:55:18,015 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38925] regionserver.StoreUtils(129): cannot split hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/cf2/534e0bc86fa24bd18c3a1abcede64e32 because midkey is the same as first or last row 2023-05-02 02:55:18,015 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38925] regionserver.HRegion(2745): Flushing 89e612e3d89dbdfa82cedca0a4b7689d 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-05-02 02:55:18,023 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38925] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/.tmp/cf1/aa034f52cd244aca9bf19174f349b546 2023-05-02 02:55:18,033 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38925] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/.tmp/cf2/c6c9b7891b384798a4f885391aa7177f 2023-05-02 02:55:18,037 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38925] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/.tmp/cf1/aa034f52cd244aca9bf19174f349b546 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/cf1/aa034f52cd244aca9bf19174f349b546 2023-05-02 02:55:18,041 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38925] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/cf1/aa034f52cd244aca9bf19174f349b546, entries=8, sequenceid=82, filesize=5.0 K 2023-05-02 02:55:18,041 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38925] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/.tmp/cf2/c6c9b7891b384798a4f885391aa7177f as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/cf2/c6c9b7891b384798a4f885391aa7177f 2023-05-02 02:55:18,045 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38925] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/cf2/c6c9b7891b384798a4f885391aa7177f, entries=8, sequenceid=82, filesize=5.0 K 2023-05-02 02:55:18,046 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38925] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 89e612e3d89dbdfa82cedca0a4b7689d in 31ms, sequenceid=82, compaction requested=true 2023-05-02 02:55:18,047 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38925] regionserver.HRegion(2446): Flush status journal for 89e612e3d89dbdfa82cedca0a4b7689d: 2023-05-02 02:55:18,047 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38925] regionserver.StoreUtils(129): cannot split hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/cf1/4a54f18026f2416ba9e253e5e5888c9e because midkey is the same as first or last row 2023-05-02 02:55:18,047 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38925] regionserver.StoreUtils(129): cannot split hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/cf2/534e0bc86fa24bd18c3a1abcede64e32 because midkey is the same as first or last row 2023-05-02 02:55:18,048 DEBUG [Listener at localhost/37119] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:18,048 DEBUG [Listener at localhost/37119] assignment.TestSplitTableRegionProcedure(438): Expected Split procedure construction failure: Region not splittable because bestSplitPoint = null, maybe table is too small for auto split. For force split, try specifying split row 2023-05-02 02:55:18,048 WARN [Listener at localhost/37119] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-02 02:55:18,050 INFO [Listener at localhost/37119] client.HBaseAdmin$15(890): Started disable of testInvalidSplitKey 2023-05-02 02:55:18,050 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testInvalidSplitKey 2023-05-02 02:55:18,051 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure2.ProcedureExecutor(1029): Stored pid=105, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testInvalidSplitKey 2023-05-02 02:55:18,053 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-05-02 02:55:18,054 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682996118054"}]},"ts":"1682996118054"} 2023-05-02 02:55:18,055 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLING in hbase:meta 2023-05-02 02:55:18,056 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testInvalidSplitKey to state=DISABLING 2023-05-02 02:55:18,057 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=89e612e3d89dbdfa82cedca0a4b7689d, UNASSIGN}] 2023-05-02 02:55:18,058 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=89e612e3d89dbdfa82cedca0a4b7689d, UNASSIGN 2023-05-02 02:55:18,058 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=89e612e3d89dbdfa82cedca0a4b7689d, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:18,058 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1682996116422.89e612e3d89dbdfa82cedca0a4b7689d.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1682996118058"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1682996118058"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682996118058"}]},"ts":"1682996118058"} 2023-05-02 02:55:18,059 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=107, ppid=106, state=RUNNABLE; CloseRegionProcedure 89e612e3d89dbdfa82cedca0a4b7689d, server=jenkins-hbase4.apache.org,38925,1682996021633}] 2023-05-02 02:55:18,154 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-05-02 02:55:18,210 ERROR [BP-376276558-172.31.14.131-1682996016523 heartbeating to localhost/127.0.0.1:38649] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-376276558-172.31.14.131-1682996016523 (Datanode Uuid 12e74f69-84cc-44e8-ac7e-14ebfb93f6fc) service to localhost/127.0.0.1:38649 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-05-02 02:55:18,211 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 89e612e3d89dbdfa82cedca0a4b7689d 2023-05-02 02:55:18,211 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 89e612e3d89dbdfa82cedca0a4b7689d, disabling compactions & flushes 2023-05-02 02:55:18,211 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testInvalidSplitKey,,1682996116422.89e612e3d89dbdfa82cedca0a4b7689d. 2023-05-02 02:55:18,211 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testInvalidSplitKey,,1682996116422.89e612e3d89dbdfa82cedca0a4b7689d. 2023-05-02 02:55:18,211 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testInvalidSplitKey,,1682996116422.89e612e3d89dbdfa82cedca0a4b7689d. after waiting 0 ms 2023-05-02 02:55:18,211 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testInvalidSplitKey,,1682996116422.89e612e3d89dbdfa82cedca0a4b7689d. 2023-05-02 02:55:18,215 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-05-02 02:55:18,216 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-02 02:55:18,216 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testInvalidSplitKey,,1682996116422.89e612e3d89dbdfa82cedca0a4b7689d. 2023-05-02 02:55:18,216 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 89e612e3d89dbdfa82cedca0a4b7689d: 2023-05-02 02:55:18,217 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 89e612e3d89dbdfa82cedca0a4b7689d 2023-05-02 02:55:18,218 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=89e612e3d89dbdfa82cedca0a4b7689d, regionState=CLOSED 2023-05-02 02:55:18,218 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1682996116422.89e612e3d89dbdfa82cedca0a4b7689d.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1682996118218"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682996118218"}]},"ts":"1682996118218"} 2023-05-02 02:55:18,220 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=107, resume processing ppid=106 2023-05-02 02:55:18,220 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=107, ppid=106, state=SUCCESS; CloseRegionProcedure 89e612e3d89dbdfa82cedca0a4b7689d, server=jenkins-hbase4.apache.org,38925,1682996021633 in 160 msec 2023-05-02 02:55:18,221 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=106, resume processing ppid=105 2023-05-02 02:55:18,222 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=106, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=89e612e3d89dbdfa82cedca0a4b7689d, UNASSIGN in 163 msec 2023-05-02 02:55:18,222 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682996118222"}]},"ts":"1682996118222"} 2023-05-02 02:55:18,223 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLED in hbase:meta 2023-05-02 02:55:18,225 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testInvalidSplitKey to state=DISABLED 2023-05-02 02:55:18,226 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey in 176 msec 2023-05-02 02:55:18,355 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-05-02 02:55:18,356 INFO [Listener at localhost/37119] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testInvalidSplitKey, procId: 105 completed 2023-05-02 02:55:18,356 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testInvalidSplitKey 2023-05-02 02:55:18,357 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure2.ProcedureExecutor(1029): Stored pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testInvalidSplitKey 2023-05-02 02:55:18,359 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-05-02 02:55:18,359 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=108, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-05-02 02:55:18,360 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1227): Checking to see if procedure is done pid=108 2023-05-02 02:55:18,363 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d 2023-05-02 02:55:18,364 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/cf1, FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/cf2, FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/recovered.edits] 2023-05-02 02:55:18,368 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/cf1/113e44b50dc14014b93fbc92b1c1025e to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/cf1/113e44b50dc14014b93fbc92b1c1025e 2023-05-02 02:55:18,369 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/cf1/34100ad5828540969babaa2f2ea76045 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/cf1/34100ad5828540969babaa2f2ea76045 2023-05-02 02:55:18,370 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/cf1/4a54f18026f2416ba9e253e5e5888c9e to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/cf1/4a54f18026f2416ba9e253e5e5888c9e 2023-05-02 02:55:18,371 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/cf1/55644b1330dc496aa5f3690bfbaa5507 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/cf1/55644b1330dc496aa5f3690bfbaa5507 2023-05-02 02:55:18,372 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/cf1/5b4a3bd9b34b4c9ca2aef0e697c63661 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/cf1/5b4a3bd9b34b4c9ca2aef0e697c63661 2023-05-02 02:55:18,373 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/cf1/a05aeeaa8ac5441a8472176dc94d2342 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/cf1/a05aeeaa8ac5441a8472176dc94d2342 2023-05-02 02:55:18,374 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/cf1/aa034f52cd244aca9bf19174f349b546 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/cf1/aa034f52cd244aca9bf19174f349b546 2023-05-02 02:55:18,376 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/cf2/175e42f3827d46ef89219b73f0e456f2 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/cf2/175e42f3827d46ef89219b73f0e456f2 2023-05-02 02:55:18,378 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/cf2/1dc04889d28f49bb9a649d4a31083f7c to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/cf2/1dc04889d28f49bb9a649d4a31083f7c 2023-05-02 02:55:18,378 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/cf2/4a4d0a972e7c4a988ba56eece569e63c to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/cf2/4a4d0a972e7c4a988ba56eece569e63c 2023-05-02 02:55:18,379 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/cf2/534e0bc86fa24bd18c3a1abcede64e32 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/cf2/534e0bc86fa24bd18c3a1abcede64e32 2023-05-02 02:55:18,380 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/cf2/60270d6a1c7f4ab18f57039b883c50d5 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/cf2/60270d6a1c7f4ab18f57039b883c50d5 2023-05-02 02:55:18,381 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/cf2/aa37ef94f30546379939765b0a5e0515 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/cf2/aa37ef94f30546379939765b0a5e0515 2023-05-02 02:55:18,382 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/cf2/c6c9b7891b384798a4f885391aa7177f to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/cf2/c6c9b7891b384798a4f885391aa7177f 2023-05-02 02:55:18,384 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/recovered.edits/85.seqid to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d/recovered.edits/85.seqid 2023-05-02 02:55:18,384 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testInvalidSplitKey/89e612e3d89dbdfa82cedca0a4b7689d 2023-05-02 02:55:18,384 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-05-02 02:55:18,386 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=108, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-05-02 02:55:18,388 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testInvalidSplitKey from hbase:meta 2023-05-02 02:55:18,392 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testInvalidSplitKey' descriptor. 2023-05-02 02:55:18,393 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=108, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-05-02 02:55:18,393 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testInvalidSplitKey' from region states. 2023-05-02 02:55:18,393 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey,,1682996116422.89e612e3d89dbdfa82cedca0a4b7689d.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1682996118393"}]},"ts":"9223372036854775807"} 2023-05-02 02:55:18,395 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-05-02 02:55:18,395 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 89e612e3d89dbdfa82cedca0a4b7689d, NAME => 'testInvalidSplitKey,,1682996116422.89e612e3d89dbdfa82cedca0a4b7689d.', STARTKEY => '', ENDKEY => ''}] 2023-05-02 02:55:18,395 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testInvalidSplitKey' as deleted. 2023-05-02 02:55:18,395 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1682996118395"}]},"ts":"9223372036854775807"} 2023-05-02 02:55:18,396 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testInvalidSplitKey state from META 2023-05-02 02:55:18,397 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=108, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-05-02 02:55:18,398 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey in 41 msec 2023-05-02 02:55:18,460 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1227): Checking to see if procedure is done pid=108 2023-05-02 02:55:18,461 INFO [Listener at localhost/37119] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testInvalidSplitKey, procId: 108 completed 2023-05-02 02:55:18,474 INFO [Listener at localhost/37119] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=369 (was 349) - Thread LEAK? -, OpenFileDescriptor=744 (was 661) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=155 (was 155), ProcessCount=170 (was 170), AvailableMemoryMB=13818 (was 13821) 2023-05-02 02:55:18,487 INFO [Listener at localhost/37119] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=369, OpenFileDescriptor=744, MaxFileDescriptor=60000, SystemLoadAverage=155, ProcessCount=170, AvailableMemoryMB=13817 2023-05-02 02:55:18,487 WARN [Listener at localhost/37119] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-02 02:55:18,489 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-05-02 02:55:18,489 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-05-02 02:55:18,491 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(1029): Stored pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-05-02 02:55:18,492 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-05-02 02:55:18,493 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-05-02 02:55:18,494 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:18,495 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003 empty. 2023-05-02 02:55:18,495 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:18,495 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-05-02 02:55:18,505 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-05-02 02:55:18,506 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(7675): creating {ENCODED => 059f75e2e092770bf7a63b32739cb003, NAME => 'testRecoveryAndDoubleExecution,,1682996118490.059f75e2e092770bf7a63b32739cb003.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRecoveryAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp 2023-05-02 02:55:18,513 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(866): Instantiated testRecoveryAndDoubleExecution,,1682996118490.059f75e2e092770bf7a63b32739cb003.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-02 02:55:18,513 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1604): Closing 059f75e2e092770bf7a63b32739cb003, disabling compactions & flushes 2023-05-02 02:55:18,513 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1626): Closing region testRecoveryAndDoubleExecution,,1682996118490.059f75e2e092770bf7a63b32739cb003. 2023-05-02 02:55:18,513 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1682996118490.059f75e2e092770bf7a63b32739cb003. 2023-05-02 02:55:18,513 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1714): Acquired close lock on testRecoveryAndDoubleExecution,,1682996118490.059f75e2e092770bf7a63b32739cb003. after waiting 0 ms 2023-05-02 02:55:18,513 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1724): Updates disabled for region testRecoveryAndDoubleExecution,,1682996118490.059f75e2e092770bf7a63b32739cb003. 2023-05-02 02:55:18,513 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1838): Closed testRecoveryAndDoubleExecution,,1682996118490.059f75e2e092770bf7a63b32739cb003. 2023-05-02 02:55:18,513 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1558): Region close journal for 059f75e2e092770bf7a63b32739cb003: 2023-05-02 02:55:18,515 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-05-02 02:55:18,516 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1682996118490.059f75e2e092770bf7a63b32739cb003.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1682996118516"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682996118516"}]},"ts":"1682996118516"} 2023-05-02 02:55:18,517 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-05-02 02:55:18,518 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-05-02 02:55:18,518 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682996118518"}]},"ts":"1682996118518"} 2023-05-02 02:55:18,519 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLING in hbase:meta 2023-05-02 02:55:18,522 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-05-02 02:55:18,523 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-05-02 02:55:18,523 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-05-02 02:55:18,523 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-05-02 02:55:18,523 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-05-02 02:55:18,523 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=059f75e2e092770bf7a63b32739cb003, ASSIGN}] 2023-05-02 02:55:18,525 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=059f75e2e092770bf7a63b32739cb003, ASSIGN 2023-05-02 02:55:18,525 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=059f75e2e092770bf7a63b32739cb003, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42019,1682996021676; forceNewPlan=false, retain=false 2023-05-02 02:55:18,676 INFO [jenkins-hbase4:41119] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-05-02 02:55:18,677 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=059f75e2e092770bf7a63b32739cb003, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:18,677 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1682996118490.059f75e2e092770bf7a63b32739cb003.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1682996118677"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1682996118677"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682996118677"}]},"ts":"1682996118677"} 2023-05-02 02:55:18,678 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=111, ppid=110, state=RUNNABLE; OpenRegionProcedure 059f75e2e092770bf7a63b32739cb003, server=jenkins-hbase4.apache.org,42019,1682996021676}] 2023-05-02 02:55:18,833 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRecoveryAndDoubleExecution,,1682996118490.059f75e2e092770bf7a63b32739cb003. 2023-05-02 02:55:18,833 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 059f75e2e092770bf7a63b32739cb003, NAME => 'testRecoveryAndDoubleExecution,,1682996118490.059f75e2e092770bf7a63b32739cb003.', STARTKEY => '', ENDKEY => ''} 2023-05-02 02:55:18,833 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-02 02:55:18,834 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:18,834 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRecoveryAndDoubleExecution,,1682996118490.059f75e2e092770bf7a63b32739cb003.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-02 02:55:18,834 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:18,834 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:18,835 INFO [StoreOpener-059f75e2e092770bf7a63b32739cb003-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:18,836 DEBUG [StoreOpener-059f75e2e092770bf7a63b32739cb003-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1 2023-05-02 02:55:18,837 DEBUG [StoreOpener-059f75e2e092770bf7a63b32739cb003-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1 2023-05-02 02:55:18,837 INFO [StoreOpener-059f75e2e092770bf7a63b32739cb003-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 059f75e2e092770bf7a63b32739cb003 columnFamilyName cf1 2023-05-02 02:55:18,838 INFO [StoreOpener-059f75e2e092770bf7a63b32739cb003-1] regionserver.HStore(310): Store=059f75e2e092770bf7a63b32739cb003/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-02 02:55:18,838 INFO [StoreOpener-059f75e2e092770bf7a63b32739cb003-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:18,839 DEBUG [StoreOpener-059f75e2e092770bf7a63b32739cb003-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2 2023-05-02 02:55:18,839 DEBUG [StoreOpener-059f75e2e092770bf7a63b32739cb003-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2 2023-05-02 02:55:18,839 INFO [StoreOpener-059f75e2e092770bf7a63b32739cb003-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 059f75e2e092770bf7a63b32739cb003 columnFamilyName cf2 2023-05-02 02:55:18,840 INFO [StoreOpener-059f75e2e092770bf7a63b32739cb003-1] regionserver.HStore(310): Store=059f75e2e092770bf7a63b32739cb003/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-02 02:55:18,841 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:18,841 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:18,842 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-02 02:55:18,843 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:18,845 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-05-02 02:55:18,845 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 059f75e2e092770bf7a63b32739cb003; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=75412588, jitterRate=0.12373512983322144}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-02 02:55:18,846 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 059f75e2e092770bf7a63b32739cb003: 2023-05-02 02:55:18,846 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,,1682996118490.059f75e2e092770bf7a63b32739cb003., pid=111, masterSystemTime=1682996118830 2023-05-02 02:55:18,848 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,,1682996118490.059f75e2e092770bf7a63b32739cb003. 2023-05-02 02:55:18,848 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRecoveryAndDoubleExecution,,1682996118490.059f75e2e092770bf7a63b32739cb003. 2023-05-02 02:55:18,848 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=059f75e2e092770bf7a63b32739cb003, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:18,848 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1682996118490.059f75e2e092770bf7a63b32739cb003.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1682996118848"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1682996118848"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1682996118848"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682996118848"}]},"ts":"1682996118848"} 2023-05-02 02:55:18,850 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=111, resume processing ppid=110 2023-05-02 02:55:18,851 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=111, ppid=110, state=SUCCESS; OpenRegionProcedure 059f75e2e092770bf7a63b32739cb003, server=jenkins-hbase4.apache.org,42019,1682996021676 in 171 msec 2023-05-02 02:55:18,852 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=110, resume processing ppid=109 2023-05-02 02:55:18,852 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=110, ppid=109, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=059f75e2e092770bf7a63b32739cb003, ASSIGN in 328 msec 2023-05-02 02:55:18,853 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-05-02 02:55:18,853 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682996118853"}]},"ts":"1682996118853"} 2023-05-02 02:55:18,854 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLED in hbase:meta 2023-05-02 02:55:18,856 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-05-02 02:55:18,857 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution in 366 msec 2023-05-02 02:55:18,998 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-02 02:55:19,000 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-05-02 02:55:19,000 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-05-02 02:55:19,000 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:19,000 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-05-02 02:55:19,000 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-05-02 02:55:19,000 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,000 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,001 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:19,001 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,001 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:19,001 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:19,001 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:19,002 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:19,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:19,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:19,002 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,002 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-02 02:55:19,002 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:19,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:19,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,002 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,002 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,003 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:19,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:19,003 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-02 02:55:19,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,003 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-02 02:55:19,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:19,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,003 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-02 02:55:19,003 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-02 02:55:19,003 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-02 02:55:19,003 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-02 02:55:19,003 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-02 02:55:19,003 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:19,003 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-02 02:55:19,003 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-02 02:55:19,003 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-02 02:55:19,003 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-02 02:55:19,003 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-02 02:55:19,003 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:19,003 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:19,003 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-02 02:55:19,003 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-02 02:55:19,003 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-02 02:55:19,003 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38925,1682996021633' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-02 02:55:19,004 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-02 02:55:19,004 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-02 02:55:19,004 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1682996118490.059f75e2e092770bf7a63b32739cb003. 2023-05-02 02:55:19,004 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1682996118490.059f75e2e092770bf7a63b32739cb003. started... 2023-05-02 02:55:19,005 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 059f75e2e092770bf7a63b32739cb003 2/2 column families, dataSize=120 B heapSize=928 B 2023-05-02 02:55:19,005 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,006 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,006 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,006 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:19,007 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,007 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:19,016 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/.tmp/cf1/84b2009af5f3401287c117850ef9fc4b 2023-05-02 02:55:19,028 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/.tmp/cf2/78391fd40923430ba069adcd774c9f4b 2023-05-02 02:55:19,032 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/.tmp/cf1/84b2009af5f3401287c117850ef9fc4b as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/84b2009af5f3401287c117850ef9fc4b 2023-05-02 02:55:19,076 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/84b2009af5f3401287c117850ef9fc4b, entries=2, sequenceid=6, filesize=4.8 K 2023-05-02 02:55:19,077 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/.tmp/cf2/78391fd40923430ba069adcd774c9f4b as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/78391fd40923430ba069adcd774c9f4b 2023-05-02 02:55:19,081 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/78391fd40923430ba069adcd774c9f4b, entries=2, sequenceid=6, filesize=4.8 K 2023-05-02 02:55:19,082 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 059f75e2e092770bf7a63b32739cb003 in 77ms, sequenceid=6, compaction requested=false 2023-05-02 02:55:19,082 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRecoveryAndDoubleExecution' 2023-05-02 02:55:19,082 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 059f75e2e092770bf7a63b32739cb003: 2023-05-02 02:55:19,082 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1682996118490.059f75e2e092770bf7a63b32739cb003. 2023-05-02 02:55:19,082 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-02 02:55:19,082 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-02 02:55:19,082 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,082 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-02 02:55:19,082 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42019,1682996021676' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-02 02:55:19,084 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,084 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:19,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:19,084 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,084 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:19,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:19,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:19,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-02 02:55:19,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:19,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:19,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:19,086 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42019,1682996021676' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-05-02 02:55:19,086 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@41c4d957[Count = 0] remaining members to acquire global barrier 2023-05-02 02:55:19,086 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-05-02 02:55:19,086 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,087 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,087 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,087 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,087 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,087 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,087 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,088 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-02 02:55:19,088 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,088 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-02 02:55:19,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,088 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-02 02:55:19,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,088 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,39081,1682996021579' in zk 2023-05-02 02:55:19,088 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-02 02:55:19,088 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-02 02:55:19,088 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-02 02:55:19,088 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-02 02:55:19,088 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,42019,1682996021676' in zk 2023-05-02 02:55:19,088 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,38925,1682996021633' in zk 2023-05-02 02:55:19,089 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-02 02:55:19,089 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:19,089 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-02 02:55:19,095 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-02 02:55:19,095 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-02 02:55:19,095 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:19,095 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:19,095 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-02 02:55:19,095 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-02 02:55:19,095 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:19,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:19,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:19,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:19,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-02 02:55:19,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:19,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:19,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:19,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-02 02:55:19,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:19,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:19,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,42019,1682996021676': 2023-05-02 02:55:19,098 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42019,1682996021676' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-05-02 02:55:19,098 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-05-02 02:55:19,098 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-02 02:55:19,098 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-02 02:55:19,098 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-05-02 02:55:19,098 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-02 02:55:19,100 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,100 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,100 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,100 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,100 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,100 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:19,100 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,100 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:19,100 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,100 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:19,100 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:19,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,100 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:19,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:19,100 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:19,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:19,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:19,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:19,101 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,101 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:19,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:19,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:19,101 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:19,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-02 02:55:19,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:19,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-02 02:55:19,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:19,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,103 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:19,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:19,103 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:19,103 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:19,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-02 02:55:19,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:19,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:19,105 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:19,105 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,105 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:19,106 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:19,106 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,106 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:19,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:19,106 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:19,106 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:19,106 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:19,106 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:19,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:19,106 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-05-02 02:55:19,106 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-02 02:55:19,106 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:19,106 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:19,106 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,106 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:19,106 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:19,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:19,106 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:19,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:19,106 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:19,106 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-05-02 02:55:19,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:19,107 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:19,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:19,106 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:19,107 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-02 02:55:19,107 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,107 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,107 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:19,107 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,107 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,207 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-02 02:55:19,208 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-02 02:55:19,219 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-02 02:55:19,220 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-05-02 02:55:19,220 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-05-02 02:55:19,220 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-05-02 02:55:19,220 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:19,220 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-05-02 02:55:19,220 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-05-02 02:55:19,221 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,221 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,222 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:19,222 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,222 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:19,222 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:19,222 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:19,222 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:19,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:19,222 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,222 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-02 02:55:19,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:19,222 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:19,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:19,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,223 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,223 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,223 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:19,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,223 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-02 02:55:19,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:19,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:19,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,223 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-02 02:55:19,223 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-02 02:55:19,223 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-05-02 02:55:19,223 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-02 02:55:19,223 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-02 02:55:19,223 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-05-02 02:55:19,223 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-05-02 02:55:19,223 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-02 02:55:19,224 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-02 02:55:19,224 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-02 02:55:19,224 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-02 02:55:19,224 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:19,223 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:19,224 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-02 02:55:19,224 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-02 02:55:19,224 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:19,224 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-02 02:55:19,224 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-02 02:55:19,224 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-02 02:55:19,224 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38925,1682996021633' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-02 02:55:19,224 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-02 02:55:19,224 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-02 02:55:19,224 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-02 02:55:19,224 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1682996118490.059f75e2e092770bf7a63b32739cb003. 2023-05-02 02:55:19,224 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1682996118490.059f75e2e092770bf7a63b32739cb003. started... 2023-05-02 02:55:19,224 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 059f75e2e092770bf7a63b32739cb003 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-02 02:55:19,227 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,227 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,227 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,227 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,227 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:19,227 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:19,233 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/.tmp/cf1/bc695fd948ff4e53b03206a746904fa3 2023-05-02 02:55:19,243 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/.tmp/cf2/9a1be768ec4e4cdbadc55ffdbfe82828 2023-05-02 02:55:19,247 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/.tmp/cf1/bc695fd948ff4e53b03206a746904fa3 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/bc695fd948ff4e53b03206a746904fa3 2023-05-02 02:55:19,251 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/bc695fd948ff4e53b03206a746904fa3, entries=10, sequenceid=19, filesize=5.0 K 2023-05-02 02:55:19,252 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/.tmp/cf2/9a1be768ec4e4cdbadc55ffdbfe82828 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/9a1be768ec4e4cdbadc55ffdbfe82828 2023-05-02 02:55:19,255 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/9a1be768ec4e4cdbadc55ffdbfe82828, entries=10, sequenceid=19, filesize=5.0 K 2023-05-02 02:55:19,255 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 059f75e2e092770bf7a63b32739cb003 in 31ms, sequenceid=19, compaction requested=false 2023-05-02 02:55:19,256 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 059f75e2e092770bf7a63b32739cb003: 2023-05-02 02:55:19,256 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1682996118490.059f75e2e092770bf7a63b32739cb003. 2023-05-02 02:55:19,257 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-02 02:55:19,257 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-02 02:55:19,257 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,257 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-02 02:55:19,257 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42019,1682996021676' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-02 02:55:19,261 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,261 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:19,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:19,261 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,261 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:19,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:19,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:19,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-02 02:55:19,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:19,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:19,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:19,263 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42019,1682996021676' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-05-02 02:55:19,263 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@406ab8f7[Count = 0] remaining members to acquire global barrier 2023-05-02 02:55:19,263 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-05-02 02:55:19,263 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,264 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,264 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,264 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,264 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,264 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,264 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,264 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-02 02:55:19,264 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-02 02:55:19,264 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,264 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-02 02:55:19,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,264 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,42019,1682996021676' in zk 2023-05-02 02:55:19,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,264 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-02 02:55:19,264 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-02 02:55:19,264 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-02 02:55:19,264 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,38925,1682996021633' in zk 2023-05-02 02:55:19,264 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-02 02:55:19,264 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,39081,1682996021579' in zk 2023-05-02 02:55:19,266 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,266 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-02 02:55:19,266 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:19,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:19,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:19,266 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-02 02:55:19,267 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-02 02:55:19,267 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-02 02:55:19,267 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:19,267 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:19,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:19,267 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-02 02:55:19,267 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-02 02:55:19,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:19,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-02 02:55:19,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:19,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:19,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:19,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-02 02:55:19,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:19,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:19,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,42019,1682996021676': 2023-05-02 02:55:19,270 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42019,1682996021676' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-05-02 02:55:19,270 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-05-02 02:55:19,270 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-02 02:55:19,270 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-02 02:55:19,270 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-05-02 02:55:19,270 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-02 02:55:19,271 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,271 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,271 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,271 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,271 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,271 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:19,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:19,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:19,271 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,271 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:19,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,272 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:19,272 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:19,271 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,271 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:19,272 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:19,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:19,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:19,272 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,272 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:19,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:19,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:19,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-02 02:55:19,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,272 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:19,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:19,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-02 02:55:19,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:19,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,274 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:19,274 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:19,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:19,274 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:19,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-02 02:55:19,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:19,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:19,277 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:19,277 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:19,277 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:19,277 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:19,277 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:19,277 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,277 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:19,277 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:19,277 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:19,277 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:19,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:19,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:19,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:19,277 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,277 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:19,277 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,277 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:19,277 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:19,277 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,277 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-05-02 02:55:19,277 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:19,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:19,277 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:19,277 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:19,277 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:19,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:19,277 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-02 02:55:19,277 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:19,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:19,278 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,278 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,278 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,278 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-05-02 02:55:19,278 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-02 02:55:19,378 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-02 02:55:19,379 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-02 02:55:19,390 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-02 02:55:19,392 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-05-02 02:55:19,392 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-05-02 02:55:19,392 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-05-02 02:55:19,392 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:19,392 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-05-02 02:55:19,392 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-05-02 02:55:19,393 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,393 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,395 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:19,395 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:19,395 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:19,395 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:19,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:19,395 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:19,395 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,395 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:19,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:19,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:19,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,396 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,396 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-02 02:55:19,396 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,396 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,396 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:19,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:19,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,396 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-02 02:55:19,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:19,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,396 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-02 02:55:19,397 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-02 02:55:19,397 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-05-02 02:55:19,397 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-02 02:55:19,397 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-05-02 02:55:19,397 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-05-02 02:55:19,397 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-02 02:55:19,397 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-02 02:55:19,397 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-02 02:55:19,397 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:19,397 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:19,397 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-02 02:55:19,397 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-02 02:55:19,397 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-02 02:55:19,397 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-02 02:55:19,397 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:19,397 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-02 02:55:19,397 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-02 02:55:19,397 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-02 02:55:19,397 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-02 02:55:19,397 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-02 02:55:19,397 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38925,1682996021633' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-02 02:55:19,397 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-02 02:55:19,397 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1682996118490.059f75e2e092770bf7a63b32739cb003. 2023-05-02 02:55:19,398 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1682996118490.059f75e2e092770bf7a63b32739cb003. started... 2023-05-02 02:55:19,398 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 059f75e2e092770bf7a63b32739cb003 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-02 02:55:19,399 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,399 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,399 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,399 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,399 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:19,399 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:19,410 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/.tmp/cf1/e5774ef6b56443f583eeef35a0086dd7 2023-05-02 02:55:19,421 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/.tmp/cf2/b9608c129d224c4fa22a6083bbb53907 2023-05-02 02:55:19,425 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/.tmp/cf1/e5774ef6b56443f583eeef35a0086dd7 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/e5774ef6b56443f583eeef35a0086dd7 2023-05-02 02:55:19,428 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/e5774ef6b56443f583eeef35a0086dd7, entries=10, sequenceid=32, filesize=5.0 K 2023-05-02 02:55:19,429 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/.tmp/cf2/b9608c129d224c4fa22a6083bbb53907 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/b9608c129d224c4fa22a6083bbb53907 2023-05-02 02:55:19,432 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/b9608c129d224c4fa22a6083bbb53907, entries=10, sequenceid=32, filesize=5.0 K 2023-05-02 02:55:19,433 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 059f75e2e092770bf7a63b32739cb003 in 35ms, sequenceid=32, compaction requested=true 2023-05-02 02:55:19,433 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 059f75e2e092770bf7a63b32739cb003: 2023-05-02 02:55:19,433 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1682996118490.059f75e2e092770bf7a63b32739cb003. 2023-05-02 02:55:19,433 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-02 02:55:19,433 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-02 02:55:19,433 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,433 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-02 02:55:19,433 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42019,1682996021676' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-02 02:55:19,434 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,434 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:19,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:19,434 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,435 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:19,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:19,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:19,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-02 02:55:19,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:19,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:19,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:19,436 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42019,1682996021676' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-05-02 02:55:19,436 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@65dd50a6[Count = 0] remaining members to acquire global barrier 2023-05-02 02:55:19,436 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-05-02 02:55:19,437 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,437 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,437 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,437 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,437 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,437 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,438 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-02 02:55:19,438 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,438 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-02 02:55:19,438 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-02 02:55:19,438 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-02 02:55:19,438 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,438 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-02 02:55:19,438 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,42019,1682996021676' in zk 2023-05-02 02:55:19,438 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-02 02:55:19,438 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,38925,1682996021633' in zk 2023-05-02 02:55:19,438 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-02 02:55:19,438 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,39081,1682996021579' in zk 2023-05-02 02:55:19,440 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,440 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-02 02:55:19,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,440 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:19,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:19,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:19,440 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-02 02:55:19,440 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-02 02:55:19,440 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:19,441 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-02 02:55:19,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:19,441 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-02 02:55:19,441 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:19,441 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-02 02:55:19,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:19,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-02 02:55:19,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:19,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:19,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:19,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-02 02:55:19,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:19,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:19,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,42019,1682996021676': 2023-05-02 02:55:19,444 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42019,1682996021676' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-05-02 02:55:19,444 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-05-02 02:55:19,444 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-02 02:55:19,444 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-02 02:55:19,444 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-05-02 02:55:19,444 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-02 02:55:19,445 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,445 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,445 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:19,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:19,445 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:19,445 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,445 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,445 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,445 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,445 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:19,445 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:19,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:19,445 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:19,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,446 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:19,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:19,446 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:19,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:19,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-02 02:55:19,446 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,446 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:19,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:19,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:19,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,446 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:19,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-02 02:55:19,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:19,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:19,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:19,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-02 02:55:19,448 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,448 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:19,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:19,449 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:19,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:19,459 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:19,459 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:19,459 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,459 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:19,459 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:19,459 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:19,459 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:19,459 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-05-02 02:55:19,459 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-02 02:55:19,459 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:19,459 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,459 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:19,459 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:19,459 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:19,459 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:19,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:19,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:19,460 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-05-02 02:55:19,459 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:19,459 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:19,460 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:19,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:19,460 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-02 02:55:19,460 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:19,460 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:19,460 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,460 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:19,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:19,460 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:19,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:19,460 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,460 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,460 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,560 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-02 02:55:19,561 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-02 02:55:19,571 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-02 02:55:19,572 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-05-02 02:55:19,572 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-05-02 02:55:19,572 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-05-02 02:55:19,572 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:19,572 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-05-02 02:55:19,572 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-05-02 02:55:19,573 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,573 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,574 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,574 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:19,574 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:19,574 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:19,574 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:19,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:19,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:19,574 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,574 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-02 02:55:19,574 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:19,574 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:19,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:19,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,575 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,575 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,575 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:19,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,575 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-02 02:55:19,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:19,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:19,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,575 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-02 02:55:19,575 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-05-02 02:55:19,575 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-02 02:55:19,576 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-02 02:55:19,576 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-05-02 02:55:19,576 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-02 02:55:19,576 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-05-02 02:55:19,576 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-02 02:55:19,576 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-02 02:55:19,576 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-02 02:55:19,576 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:19,576 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:19,576 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-02 02:55:19,576 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-02 02:55:19,576 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-02 02:55:19,576 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-02 02:55:19,576 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:19,576 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-02 02:55:19,576 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-02 02:55:19,576 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-02 02:55:19,576 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38925,1682996021633' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-02 02:55:19,576 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1682996118490.059f75e2e092770bf7a63b32739cb003. 2023-05-02 02:55:19,576 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-02 02:55:19,576 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-02 02:55:19,576 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1682996118490.059f75e2e092770bf7a63b32739cb003. started... 2023-05-02 02:55:19,577 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 059f75e2e092770bf7a63b32739cb003 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-02 02:55:19,578 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,578 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,578 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,578 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:19,578 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,578 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:19,586 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/.tmp/cf1/8f67930f70934c80893ea92e3da97da1 2023-05-02 02:55:19,597 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/.tmp/cf2/902bebca210f45908626f299bace6411 2023-05-02 02:55:19,601 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/.tmp/cf1/8f67930f70934c80893ea92e3da97da1 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/8f67930f70934c80893ea92e3da97da1 2023-05-02 02:55:19,604 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/8f67930f70934c80893ea92e3da97da1, entries=10, sequenceid=45, filesize=5.0 K 2023-05-02 02:55:19,605 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/.tmp/cf2/902bebca210f45908626f299bace6411 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/902bebca210f45908626f299bace6411 2023-05-02 02:55:19,608 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/902bebca210f45908626f299bace6411, entries=10, sequenceid=45, filesize=5.0 K 2023-05-02 02:55:19,609 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 059f75e2e092770bf7a63b32739cb003 in 32ms, sequenceid=45, compaction requested=true 2023-05-02 02:55:19,609 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 059f75e2e092770bf7a63b32739cb003: 2023-05-02 02:55:19,609 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1682996118490.059f75e2e092770bf7a63b32739cb003. 2023-05-02 02:55:19,609 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-02 02:55:19,609 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-02 02:55:19,609 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,609 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-02 02:55:19,609 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42019,1682996021676' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-02 02:55:19,612 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,612 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:19,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:19,612 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,612 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:19,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:19,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:19,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-02 02:55:19,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:19,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:19,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:19,614 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42019,1682996021676' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-05-02 02:55:19,614 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3355c8bd[Count = 0] remaining members to acquire global barrier 2023-05-02 02:55:19,614 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-05-02 02:55:19,614 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,615 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,615 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,615 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,615 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,615 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,615 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,615 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,615 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-02 02:55:19,615 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-02 02:55:19,615 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-02 02:55:19,615 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-02 02:55:19,615 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-02 02:55:19,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,615 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,39081,1682996021579' in zk 2023-05-02 02:55:19,615 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-02 02:55:19,615 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-02 02:55:19,615 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,38925,1682996021633' in zk 2023-05-02 02:55:19,615 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,42019,1682996021676' in zk 2023-05-02 02:55:19,617 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-02 02:55:19,617 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:19,617 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-02 02:55:19,617 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:19,617 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-02 02:55:19,617 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-02 02:55:19,618 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,618 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-02 02:55:19,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,618 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:19,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:19,618 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-02 02:55:19,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:19,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:19,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:19,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-02 02:55:19,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:19,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:19,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:19,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-02 02:55:19,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:19,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:19,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,42019,1682996021676': 2023-05-02 02:55:19,620 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42019,1682996021676' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-05-02 02:55:19,620 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-05-02 02:55:19,620 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-02 02:55:19,620 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-02 02:55:19,620 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-05-02 02:55:19,620 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-02 02:55:19,623 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,623 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:19,623 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,623 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:19,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,623 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,623 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,623 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,623 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:19,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:19,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,624 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:19,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:19,623 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,623 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:19,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,624 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:19,624 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:19,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:19,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:19,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,624 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:19,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:19,624 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-02 02:55:19,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,625 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:19,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:19,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-02 02:55:19,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:19,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:19,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:19,626 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:19,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-02 02:55:19,626 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:19,626 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:19,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:19,629 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:19,629 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:19,629 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:19,629 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:19,629 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:19,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:19,629 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:19,629 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,629 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:19,629 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,629 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-05-02 02:55:19,629 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-02 02:55:19,629 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:19,629 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:19,629 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:19,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:19,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:19,629 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:19,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:19,629 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:19,629 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:19,630 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-05-02 02:55:19,630 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:19,630 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:19,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:19,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:19,630 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-02 02:55:19,630 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,630 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:19,630 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,630 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,630 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:19,630 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,630 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,730 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-02 02:55:19,731 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-02 02:55:19,743 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-02 02:55:19,745 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-05-02 02:55:19,745 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-05-02 02:55:19,745 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-05-02 02:55:19,745 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:19,745 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-05-02 02:55:19,745 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-05-02 02:55:19,746 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,746 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,747 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:19,747 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:19,747 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:19,747 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,747 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:19,747 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:19,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:19,747 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:19,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:19,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:19,747 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,747 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-02 02:55:19,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,748 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,748 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,748 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:19,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,748 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-02 02:55:19,749 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-05-02 02:55:19,749 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-02 02:55:19,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:19,749 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-02 02:55:19,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:19,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,749 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-02 02:55:19,749 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:19,749 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-02 02:55:19,749 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-02 02:55:19,749 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-02 02:55:19,749 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-02 02:55:19,749 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-05-02 02:55:19,749 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-05-02 02:55:19,749 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-02 02:55:19,749 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-02 02:55:19,749 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-02 02:55:19,749 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:19,749 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-02 02:55:19,749 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:19,749 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-02 02:55:19,750 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-02 02:55:19,750 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-02 02:55:19,750 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38925,1682996021633' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-02 02:55:19,750 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-02 02:55:19,750 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-02 02:55:19,750 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1682996118490.059f75e2e092770bf7a63b32739cb003. 2023-05-02 02:55:19,750 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1682996118490.059f75e2e092770bf7a63b32739cb003. started... 2023-05-02 02:55:19,750 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 059f75e2e092770bf7a63b32739cb003 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-02 02:55:19,750 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,756 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,756 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,756 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:19,756 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,756 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:19,765 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/.tmp/cf1/0837374a7f2a4c82bfa97443781ee242 2023-05-02 02:55:19,780 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/.tmp/cf2/4ddd3ec587eb4d9f9c2d89d53cdc963f 2023-05-02 02:55:19,783 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/.tmp/cf1/0837374a7f2a4c82bfa97443781ee242 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/0837374a7f2a4c82bfa97443781ee242 2023-05-02 02:55:19,787 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/0837374a7f2a4c82bfa97443781ee242, entries=10, sequenceid=58, filesize=5.0 K 2023-05-02 02:55:19,787 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/.tmp/cf2/4ddd3ec587eb4d9f9c2d89d53cdc963f as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/4ddd3ec587eb4d9f9c2d89d53cdc963f 2023-05-02 02:55:19,790 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/4ddd3ec587eb4d9f9c2d89d53cdc963f, entries=10, sequenceid=58, filesize=5.0 K 2023-05-02 02:55:19,791 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 059f75e2e092770bf7a63b32739cb003 in 41ms, sequenceid=58, compaction requested=true 2023-05-02 02:55:19,791 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 059f75e2e092770bf7a63b32739cb003: 2023-05-02 02:55:19,791 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1682996118490.059f75e2e092770bf7a63b32739cb003. 2023-05-02 02:55:19,791 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-02 02:55:19,791 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-02 02:55:19,791 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,792 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-02 02:55:19,792 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42019,1682996021676' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-02 02:55:19,796 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,796 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:19,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:19,796 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,796 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:19,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:19,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:19,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-02 02:55:19,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:19,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:19,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:19,798 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42019,1682996021676' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-05-02 02:55:19,798 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@54bca1e1[Count = 0] remaining members to acquire global barrier 2023-05-02 02:55:19,798 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-05-02 02:55:19,798 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,799 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,799 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,799 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,799 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,799 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,799 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-02 02:55:19,800 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-02 02:55:19,800 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-02 02:55:19,800 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,800 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-02 02:55:19,800 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,800 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,39081,1682996021579' in zk 2023-05-02 02:55:19,800 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-02 02:55:19,800 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,38925,1682996021633' in zk 2023-05-02 02:55:19,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,800 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-02 02:55:19,800 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-02 02:55:19,800 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,42019,1682996021676' in zk 2023-05-02 02:55:19,802 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-02 02:55:19,802 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:19,802 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-02 02:55:19,803 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-02 02:55:19,803 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,803 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-02 02:55:19,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:19,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:19,803 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:19,803 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:19,803 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-02 02:55:19,803 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-02 02:55:19,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:19,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:19,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-02 02:55:19,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:19,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:19,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:19,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-02 02:55:19,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:19,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:19,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,42019,1682996021676': 2023-05-02 02:55:19,806 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42019,1682996021676' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-05-02 02:55:19,806 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-05-02 02:55:19,806 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-02 02:55:19,806 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-02 02:55:19,806 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-05-02 02:55:19,806 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-02 02:55:19,808 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,808 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,808 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,808 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,808 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,808 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:19,808 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,808 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:19,808 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:19,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:19,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:19,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:19,808 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:19,808 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,808 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:19,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:19,809 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:19,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:19,809 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,809 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:19,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:19,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-02 02:55:19,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,809 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:19,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:19,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-02 02:55:19,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:19,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:19,810 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:19,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:19,811 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-02 02:55:19,811 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:19,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:19,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:19,813 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:19,813 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:19,813 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:19,813 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:19,813 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:19,813 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:19,813 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:19,813 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:19,813 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,813 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:19,813 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:19,813 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:19,813 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,813 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:19,813 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:19,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:19,813 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-05-02 02:55:19,814 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:19,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:19,813 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:19,814 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-02 02:55:19,814 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,814 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:19,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:19,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:19,814 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,814 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:19,814 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,814 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,814 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-05-02 02:55:19,814 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-02 02:55:19,815 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:19,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:19,914 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-02 02:55:19,916 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-02 02:55:19,926 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-02 02:55:19,927 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-05-02 02:55:19,928 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-05-02 02:55:19,928 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-05-02 02:55:19,928 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:19,928 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-05-02 02:55:19,928 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-05-02 02:55:19,928 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,928 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,930 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:19,930 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:19,930 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,930 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:19,930 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:19,930 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:19,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:19,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:19,930 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:19,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:19,931 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,931 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-02 02:55:19,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,931 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,931 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,931 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:19,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:19,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,932 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-02 02:55:19,932 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-02 02:55:19,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:19,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,932 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-02 02:55:19,932 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-05-02 02:55:19,932 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-02 02:55:19,932 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-05-02 02:55:19,932 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-02 02:55:19,932 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:19,932 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-02 02:55:19,932 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-02 02:55:19,932 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-02 02:55:19,932 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:19,932 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-05-02 02:55:19,932 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-02 02:55:19,932 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-02 02:55:19,932 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38925,1682996021633' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-02 02:55:19,932 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-02 02:55:19,932 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-02 02:55:19,933 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-02 02:55:19,933 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-02 02:55:19,933 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:19,933 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-02 02:55:19,933 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-02 02:55:19,933 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-02 02:55:19,933 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1682996118490.059f75e2e092770bf7a63b32739cb003. 2023-05-02 02:55:19,933 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1682996118490.059f75e2e092770bf7a63b32739cb003. started... 2023-05-02 02:55:19,933 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 059f75e2e092770bf7a63b32739cb003 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-02 02:55:19,934 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,934 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,934 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,934 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:19,934 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,934 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:19,941 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/.tmp/cf1/b1149ff9cf704fc0b06d4a93d73229dc 2023-05-02 02:55:19,953 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/.tmp/cf2/b3d0ee38ca69423f9a6877578518bc76 2023-05-02 02:55:19,957 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/.tmp/cf1/b1149ff9cf704fc0b06d4a93d73229dc as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/b1149ff9cf704fc0b06d4a93d73229dc 2023-05-02 02:55:19,960 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/b1149ff9cf704fc0b06d4a93d73229dc, entries=10, sequenceid=71, filesize=5.0 K 2023-05-02 02:55:19,960 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/.tmp/cf2/b3d0ee38ca69423f9a6877578518bc76 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/b3d0ee38ca69423f9a6877578518bc76 2023-05-02 02:55:19,963 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/b3d0ee38ca69423f9a6877578518bc76, entries=10, sequenceid=71, filesize=5.0 K 2023-05-02 02:55:19,964 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 059f75e2e092770bf7a63b32739cb003 in 31ms, sequenceid=71, compaction requested=true 2023-05-02 02:55:19,964 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 059f75e2e092770bf7a63b32739cb003: 2023-05-02 02:55:19,964 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1682996118490.059f75e2e092770bf7a63b32739cb003. 2023-05-02 02:55:19,964 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-02 02:55:19,964 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-02 02:55:19,965 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,965 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-02 02:55:19,965 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42019,1682996021676' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-02 02:55:19,966 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,966 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:19,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:19,966 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,966 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:19,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:19,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:19,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-02 02:55:19,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:19,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:19,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:19,968 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42019,1682996021676' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-05-02 02:55:19,968 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@41041681[Count = 0] remaining members to acquire global barrier 2023-05-02 02:55:19,968 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-05-02 02:55:19,968 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,969 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,969 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,969 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,969 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,969 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,969 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-02 02:55:19,969 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,969 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-02 02:55:19,969 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-02 02:55:19,969 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,969 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,39081,1682996021579' in zk 2023-05-02 02:55:19,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,969 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-02 02:55:19,970 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-02 02:55:19,970 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,38925,1682996021633' in zk 2023-05-02 02:55:19,970 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-02 02:55:19,970 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-02 02:55:19,970 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,42019,1682996021676' in zk 2023-05-02 02:55:19,973 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-02 02:55:19,973 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:19,973 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-02 02:55:19,974 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-02 02:55:19,974 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-02 02:55:19,974 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:19,974 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:19,974 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-02 02:55:19,974 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-02 02:55:19,974 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:19,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:19,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:19,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:19,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-02 02:55:19,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:19,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:19,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:19,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-02 02:55:19,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:19,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:19,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,42019,1682996021676': 2023-05-02 02:55:19,977 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42019,1682996021676' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-05-02 02:55:19,977 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-05-02 02:55:19,977 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-02 02:55:19,977 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-02 02:55:19,977 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-05-02 02:55:19,977 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-02 02:55:19,978 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,978 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,978 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:19,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:19,978 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:19,978 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,978 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,978 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,979 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:19,979 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:19,978 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:19,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,979 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:19,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-02 02:55:19,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,979 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:19,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:19,979 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:19,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:19,979 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,979 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:19,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:19,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:19,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-02 02:55:19,980 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:19,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:19,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:19,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:19,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-02 02:55:19,981 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:19,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:19,982 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,982 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:19,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:19,987 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:19,987 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:19,987 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:19,987 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:19,987 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:19,987 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:19,987 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:19,987 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:19,987 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:19,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:19,987 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:19,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:19,987 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,987 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:19,987 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,987 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:19,988 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,988 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:19,988 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,987 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-05-02 02:55:19,988 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-02 02:55:19,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:19,988 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:19,988 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:19,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:19,988 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:19,988 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,988 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-02 02:55:19,988 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-05-02 02:55:19,988 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-02 02:55:19,989 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:19,989 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:19,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:19,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:20,088 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-02 02:55:20,089 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-02 02:55:20,348 WARN [Listener at localhost/37119] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-05-02 02:55:20,349 DEBUG [Listener at localhost/37119] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:20,350 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(1029): Stored pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:20,351 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:20,351 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:20,351 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:20,351 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-02 02:55:20,352 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:20,352 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-02 02:55:20,352 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-02 02:55:20,600 INFO [Listener at localhost/37119] procedure.MasterProcedureTestingUtility(454): Restart 0 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:20,600 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-02 02:55:20,600 INFO [Listener at localhost/37119] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-02 02:55:20,600 INFO [Listener at localhost/37119] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-02 02:55:20,600 ERROR [Listener at localhost/37119] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-9,5,PEWorkerGroup] Thread[HFileArchiver-10,5,PEWorkerGroup] Thread[HFileArchiver-11,5,PEWorkerGroup] Thread[HFileArchiver-12,5,PEWorkerGroup] 2023-05-02 02:55:20,600 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-02 02:55:20,600 INFO [Listener at localhost/37119] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-02 02:55:20,600 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-02 02:55:20,601 INFO [Listener at localhost/37119] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-02 02:55:20,601 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-02 02:55:20,603 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-05-02 02:55:20,603 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-02 02:55:20,604 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-02 02:55:20,604 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-02 02:55:20,604 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-02 02:55:20,604 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:55:20,605 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=084b1607f7ed8615120a04e4729fbafa, daughterA=ea7683cd3436a62eb16418a3c875d563, daughterB=408e891cab9b919f4038b233ba34779d 2023-05-02 02:55:20,605 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:55:20,605 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:55:20,605 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-02 02:55:20,606 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:55:20,606 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-02 02:55:20,606 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-02 02:55:20,606 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-02 02:55:20,607 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:55:20,607 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, ASSIGN 2023-05-02 02:55:20,607 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-05-02 02:55:20,607 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-05-02 02:55:20,607 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-02 02:55:20,608 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=afa57eddfa2d26bf35b12dc022351e78, daughterA=b91a201af734ea9cb01587589c96e908, daughterB=292a312492e2395ec0ca83a160894be9 2023-05-02 02:55:20,608 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-02 02:55:20,608 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-02 02:55:20,608 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:55:20,609 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=8af7d4111b21c70efea08f7beab73a9f, daughterA=bb895e1df53cda46bfcb5f94fe1710c9, daughterB=024f19513d3a9ded6c060054f1607e68 2023-05-02 02:55:20,609 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:55:20,609 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:55:20,609 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-02 02:55:20,610 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-02 02:55:20,610 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,39081,1682996021579, table=testRollbackForSplitTableRegionWithReplica, region=1b91efe506b89bc57ba68287b021be4f is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=74b19512a655e6f00fc418d81fba16c8, daughterA=bfa227908c6548a4187e4deb87e808a6, daughterB=da50a5f8e634bd92f99eebe500206397 2023-05-02 02:55:20,610 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-02 02:55:20,611 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-02 02:55:20,611 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-05-02 02:55:20,611 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-05-02 02:55:20,611 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-05-02 02:55:20,612 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-05-02 02:55:20,612 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:20,612 DEBUG [Listener at localhost/37119] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 held the lock before restarting, call acquireLock to restore it. 2023-05-02 02:55:20,612 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:20,612 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:20,612 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:20,613 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 11 msec 2023-05-02 02:55:20,613 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-02 02:55:20,614 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,38925,1682996021633, table=hbase:meta, region=1588230740 2023-05-02 02:55:20,614 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682996150614 2023-05-02 02:55:20,615 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(1517): Joining cluster... 2023-05-02 02:55:20,616 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:20,618 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=5c823d6f15ad7b8fe82f9ae4beee6513, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42019,1682996021676, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676, openSeqNum=2 2023-05-02 02:55:20,618 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=059f75e2e092770bf7a63b32739cb003, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42019,1682996021676, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676, openSeqNum=2 2023-05-02 02:55:20,618 INFO [Listener at localhost/37119] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-02 02:55:20,618 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682996180618 2023-05-02 02:55:20,618 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682996240618 2023-05-02 02:55:20,618 INFO [Listener at localhost/37119] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-02 02:55:20,619 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-02 02:55:20,620 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-02 02:55:20,620 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:20,620 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-02 02:55:20,620 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-02 02:55:20,869 INFO [Listener at localhost/37119] procedure.MasterProcedureTestingUtility(454): Restart 1 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:20,869 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-02 02:55:20,869 INFO [Listener at localhost/37119] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-02 02:55:20,869 INFO [Listener at localhost/37119] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-02 02:55:20,869 ERROR [Listener at localhost/37119] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-02 02:55:20,869 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-02 02:55:20,869 INFO [Listener at localhost/37119] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-02 02:55:20,869 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-02 02:55:20,869 INFO [Listener at localhost/37119] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-02 02:55:20,870 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-02 02:55:20,871 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-05-02 02:55:20,871 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-02 02:55:20,872 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-02 02:55:20,872 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-02 02:55:20,872 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-02 02:55:20,872 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:55:20,872 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=084b1607f7ed8615120a04e4729fbafa, daughterA=ea7683cd3436a62eb16418a3c875d563, daughterB=408e891cab9b919f4038b233ba34779d 2023-05-02 02:55:20,872 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:55:20,873 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:55:20,873 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-02 02:55:20,873 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:55:20,873 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-02 02:55:20,873 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-02 02:55:20,873 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-02 02:55:20,874 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:55:20,874 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, ASSIGN 2023-05-02 02:55:20,874 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-05-02 02:55:20,874 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-05-02 02:55:20,874 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-02 02:55:20,874 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=afa57eddfa2d26bf35b12dc022351e78, daughterA=b91a201af734ea9cb01587589c96e908, daughterB=292a312492e2395ec0ca83a160894be9 2023-05-02 02:55:20,874 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-02 02:55:20,874 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-02 02:55:20,875 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:55:20,875 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=8af7d4111b21c70efea08f7beab73a9f, daughterA=bb895e1df53cda46bfcb5f94fe1710c9, daughterB=024f19513d3a9ded6c060054f1607e68 2023-05-02 02:55:20,875 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:55:20,875 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:55:20,875 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-02 02:55:20,875 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-02 02:55:20,875 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,39081,1682996021579, table=testRollbackForSplitTableRegionWithReplica, region=1b91efe506b89bc57ba68287b021be4f is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=74b19512a655e6f00fc418d81fba16c8, daughterA=bfa227908c6548a4187e4deb87e808a6, daughterB=da50a5f8e634bd92f99eebe500206397 2023-05-02 02:55:20,876 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-02 02:55:20,876 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-02 02:55:20,876 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-05-02 02:55:20,876 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-05-02 02:55:20,876 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-05-02 02:55:20,876 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-05-02 02:55:20,876 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:20,876 DEBUG [Listener at localhost/37119] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 held the lock before restarting, call acquireLock to restore it. 2023-05-02 02:55:20,877 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:20,877 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:20,877 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:20,877 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-05-02 02:55:20,877 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-02 02:55:20,878 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,38925,1682996021633, table=hbase:meta, region=1588230740 2023-05-02 02:55:20,878 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682996150878 2023-05-02 02:55:20,880 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:20,880 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(1517): Joining cluster... 2023-05-02 02:55:20,882 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=5c823d6f15ad7b8fe82f9ae4beee6513, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42019,1682996021676, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676, openSeqNum=2 2023-05-02 02:55:20,882 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=059f75e2e092770bf7a63b32739cb003, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42019,1682996021676, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676, openSeqNum=2 2023-05-02 02:55:20,882 INFO [Listener at localhost/37119] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-02 02:55:20,882 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682996180882 2023-05-02 02:55:20,882 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682996240882 2023-05-02 02:55:20,883 INFO [Listener at localhost/37119] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-05-02 02:55:20,883 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-02 02:55:20,885 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=059f75e2e092770bf7a63b32739cb003, UNASSIGN}] 2023-05-02 02:55:20,885 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-02 02:55:20,885 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:20,885 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-02 02:55:20,885 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-02 02:55:21,133 INFO [Listener at localhost/37119] procedure.MasterProcedureTestingUtility(454): Restart 2 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:21,133 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-02 02:55:21,133 INFO [Listener at localhost/37119] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-02 02:55:21,133 INFO [Listener at localhost/37119] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-02 02:55:21,133 ERROR [Listener at localhost/37119] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-02 02:55:21,133 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-02 02:55:21,133 INFO [Listener at localhost/37119] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-02 02:55:21,133 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-02 02:55:21,133 INFO [Listener at localhost/37119] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-02 02:55:21,134 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-02 02:55:21,136 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-05-02 02:55:21,136 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-02 02:55:21,137 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-02 02:55:21,137 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-02 02:55:21,137 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-02 02:55:21,137 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:55:21,137 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=084b1607f7ed8615120a04e4729fbafa, daughterA=ea7683cd3436a62eb16418a3c875d563, daughterB=408e891cab9b919f4038b233ba34779d 2023-05-02 02:55:21,137 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:55:21,138 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:55:21,138 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-02 02:55:21,138 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:55:21,138 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-02 02:55:21,138 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-02 02:55:21,138 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-02 02:55:21,138 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:55:21,139 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, ASSIGN 2023-05-02 02:55:21,139 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-05-02 02:55:21,139 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-05-02 02:55:21,139 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-02 02:55:21,139 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=afa57eddfa2d26bf35b12dc022351e78, daughterA=b91a201af734ea9cb01587589c96e908, daughterB=292a312492e2395ec0ca83a160894be9 2023-05-02 02:55:21,139 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-02 02:55:21,139 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-02 02:55:21,140 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:55:21,140 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=8af7d4111b21c70efea08f7beab73a9f, daughterA=bb895e1df53cda46bfcb5f94fe1710c9, daughterB=024f19513d3a9ded6c060054f1607e68 2023-05-02 02:55:21,140 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:55:21,140 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:55:21,141 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-02 02:55:21,141 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-02 02:55:21,141 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,39081,1682996021579, table=testRollbackForSplitTableRegionWithReplica, region=1b91efe506b89bc57ba68287b021be4f is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=74b19512a655e6f00fc418d81fba16c8, daughterA=bfa227908c6548a4187e4deb87e808a6, daughterB=da50a5f8e634bd92f99eebe500206397 2023-05-02 02:55:21,142 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-02 02:55:21,142 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-02 02:55:21,142 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-05-02 02:55:21,142 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-05-02 02:55:21,143 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-05-02 02:55:21,143 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-05-02 02:55:21,143 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:21,143 DEBUG [Listener at localhost/37119] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 held the lock before restarting, call acquireLock to restore it. 2023-05-02 02:55:21,143 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:21,143 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:21,144 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:21,144 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-05-02 02:55:21,144 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-02 02:55:21,145 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,38925,1682996021633, table=hbase:meta, region=1588230740 2023-05-02 02:55:21,148 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682996151148 2023-05-02 02:55:21,148 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(1517): Joining cluster... 2023-05-02 02:55:21,149 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:21,152 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=5c823d6f15ad7b8fe82f9ae4beee6513, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42019,1682996021676, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676, openSeqNum=2 2023-05-02 02:55:21,153 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=059f75e2e092770bf7a63b32739cb003, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42019,1682996021676, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676, openSeqNum=2 2023-05-02 02:55:21,153 INFO [Listener at localhost/37119] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-02 02:55:21,153 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682996181153 2023-05-02 02:55:21,153 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682996241153 2023-05-02 02:55:21,153 INFO [Listener at localhost/37119] assignment.AssignmentManager(1536): Joined the cluster in 5 msec 2023-05-02 02:55:21,154 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=059f75e2e092770bf7a63b32739cb003, UNASSIGN}] 2023-05-02 02:55:21,154 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-02 02:55:21,156 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=059f75e2e092770bf7a63b32739cb003, UNASSIGN 2023-05-02 02:55:21,157 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=059f75e2e092770bf7a63b32739cb003, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:21,157 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1682996118490.059f75e2e092770bf7a63b32739cb003.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1682996121157"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1682996121157"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682996121157"}]},"ts":"1682996121157"} 2023-05-02 02:55:21,158 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=114, ppid=113, state=RUNNABLE; CloseRegionProcedure 059f75e2e092770bf7a63b32739cb003, server=jenkins-hbase4.apache.org,42019,1682996021676}] 2023-05-02 02:55:21,209 ERROR [BP-376276558-172.31.14.131-1682996016523 heartbeating to localhost/127.0.0.1:38649] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-376276558-172.31.14.131-1682996016523 (Datanode Uuid 2b091f00-29ea-476e-ba4f-8ff3dee54078) service to localhost/127.0.0.1:38649 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-05-02 02:55:21,310 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:21,310 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 059f75e2e092770bf7a63b32739cb003, disabling compactions & flushes 2023-05-02 02:55:21,310 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRecoveryAndDoubleExecution,,1682996118490.059f75e2e092770bf7a63b32739cb003. 2023-05-02 02:55:21,310 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1682996118490.059f75e2e092770bf7a63b32739cb003. 2023-05-02 02:55:21,311 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRecoveryAndDoubleExecution,,1682996118490.059f75e2e092770bf7a63b32739cb003. after waiting 0 ms 2023-05-02 02:55:21,311 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRecoveryAndDoubleExecution,,1682996118490.059f75e2e092770bf7a63b32739cb003. 2023-05-02 02:55:21,311 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 059f75e2e092770bf7a63b32739cb003 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-05-02 02:55:21,319 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/.tmp/cf1/21a49995122e48a493b6d0a8abb3ff52 2023-05-02 02:55:21,330 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/.tmp/cf2/b8675e8553444d2bbde903dca91aba1e 2023-05-02 02:55:21,333 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/.tmp/cf1/21a49995122e48a493b6d0a8abb3ff52 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/21a49995122e48a493b6d0a8abb3ff52 2023-05-02 02:55:21,337 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/21a49995122e48a493b6d0a8abb3ff52, entries=8, sequenceid=82, filesize=5.0 K 2023-05-02 02:55:21,337 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/.tmp/cf2/b8675e8553444d2bbde903dca91aba1e as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/b8675e8553444d2bbde903dca91aba1e 2023-05-02 02:55:21,341 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/b8675e8553444d2bbde903dca91aba1e, entries=8, sequenceid=82, filesize=5.0 K 2023-05-02 02:55:21,342 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 059f75e2e092770bf7a63b32739cb003 in 31ms, sequenceid=82, compaction requested=true 2023-05-02 02:55:21,345 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-05-02 02:55:21,346 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-02 02:55:21,347 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRecoveryAndDoubleExecution,,1682996118490.059f75e2e092770bf7a63b32739cb003. 2023-05-02 02:55:21,347 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 059f75e2e092770bf7a63b32739cb003: 2023-05-02 02:55:21,349 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:21,349 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=059f75e2e092770bf7a63b32739cb003, regionState=CLOSED 2023-05-02 02:55:21,349 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1682996118490.059f75e2e092770bf7a63b32739cb003.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1682996121349"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682996121349"}]},"ts":"1682996121349"} 2023-05-02 02:55:21,352 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=114, resume processing ppid=113 2023-05-02 02:55:21,352 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 059f75e2e092770bf7a63b32739cb003, server=jenkins-hbase4.apache.org,42019,1682996021676 in 193 msec 2023-05-02 02:55:21,353 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=113, resume processing ppid=112 2023-05-02 02:55:21,353 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=059f75e2e092770bf7a63b32739cb003, UNASSIGN in 198 msec 2023-05-02 02:55:21,355 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-02 02:55:21,356 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:21,356 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-02 02:55:21,356 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-02 02:55:21,403 INFO [Listener at localhost/37119] procedure.MasterProcedureTestingUtility(454): Restart 3 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:21,404 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-02 02:55:21,404 INFO [Listener at localhost/37119] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-02 02:55:21,404 INFO [Listener at localhost/37119] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-02 02:55:21,404 ERROR [Listener at localhost/37119] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-02 02:55:21,404 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-02 02:55:21,404 INFO [Listener at localhost/37119] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-02 02:55:21,404 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-02 02:55:21,404 INFO [Listener at localhost/37119] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-02 02:55:21,405 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-02 02:55:21,406 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-05-02 02:55:21,406 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-02 02:55:21,406 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-02 02:55:21,406 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-02 02:55:21,407 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-02 02:55:21,407 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-02 02:55:21,407 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-02 02:55:21,407 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:55:21,407 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=084b1607f7ed8615120a04e4729fbafa, daughterA=ea7683cd3436a62eb16418a3c875d563, daughterB=408e891cab9b919f4038b233ba34779d 2023-05-02 02:55:21,407 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:55:21,407 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:55:21,408 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-02 02:55:21,408 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:55:21,408 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-02 02:55:21,408 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-02 02:55:21,408 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-02 02:55:21,409 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:55:21,409 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, ASSIGN 2023-05-02 02:55:21,409 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-05-02 02:55:21,409 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-05-02 02:55:21,409 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-02 02:55:21,409 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=afa57eddfa2d26bf35b12dc022351e78, daughterA=b91a201af734ea9cb01587589c96e908, daughterB=292a312492e2395ec0ca83a160894be9 2023-05-02 02:55:21,409 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-02 02:55:21,409 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-02 02:55:21,410 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:55:21,410 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=8af7d4111b21c70efea08f7beab73a9f, daughterA=bb895e1df53cda46bfcb5f94fe1710c9, daughterB=024f19513d3a9ded6c060054f1607e68 2023-05-02 02:55:21,410 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:55:21,410 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:55:21,410 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-02 02:55:21,410 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-02 02:55:21,411 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,39081,1682996021579, table=testRollbackForSplitTableRegionWithReplica, region=1b91efe506b89bc57ba68287b021be4f is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=74b19512a655e6f00fc418d81fba16c8, daughterA=bfa227908c6548a4187e4deb87e808a6, daughterB=da50a5f8e634bd92f99eebe500206397 2023-05-02 02:55:21,411 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-02 02:55:21,411 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-02 02:55:21,411 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-05-02 02:55:21,411 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-05-02 02:55:21,411 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-05-02 02:55:21,411 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-05-02 02:55:21,411 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:21,412 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=059f75e2e092770bf7a63b32739cb003, UNASSIGN 2023-05-02 02:55:21,412 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 059f75e2e092770bf7a63b32739cb003, server=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:21,412 DEBUG [Listener at localhost/37119] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 held the lock before restarting, call acquireLock to restore it. 2023-05-02 02:55:21,412 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:21,412 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:21,412 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:21,412 DEBUG [Listener at localhost/37119] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=059f75e2e092770bf7a63b32739cb003, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-02 02:55:21,412 DEBUG [Listener at localhost/37119] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 059f75e2e092770bf7a63b32739cb003, server=jenkins-hbase4.apache.org,42019,1682996021676 didn't hold the lock before restarting, skip acquiring lock. 2023-05-02 02:55:21,412 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-05-02 02:55:21,413 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-02 02:55:21,413 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,38925,1682996021633, table=hbase:meta, region=1588230740 2023-05-02 02:55:21,414 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682996151414 2023-05-02 02:55:21,415 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:21,415 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(1517): Joining cluster... 2023-05-02 02:55:21,418 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=5c823d6f15ad7b8fe82f9ae4beee6513, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42019,1682996021676, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676, openSeqNum=2 2023-05-02 02:55:21,419 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=059f75e2e092770bf7a63b32739cb003, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,42019,1682996021676, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676, openSeqNum=2 2023-05-02 02:55:21,419 INFO [Listener at localhost/37119] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-02 02:55:21,419 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682996181419 2023-05-02 02:55:21,419 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682996241419 2023-05-02 02:55:21,419 INFO [Listener at localhost/37119] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-02 02:55:21,422 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-02 02:55:21,431 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=059f75e2e092770bf7a63b32739cb003, threads=14 2023-05-02 02:55:21,431 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/4ddd3ec587eb4d9f9c2d89d53cdc963f for region: 059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:21,434 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/78391fd40923430ba069adcd774c9f4b for region: 059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:21,435 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/902bebca210f45908626f299bace6411 for region: 059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:21,435 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/9a1be768ec4e4cdbadc55ffdbfe82828 for region: 059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:21,435 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/b8675e8553444d2bbde903dca91aba1e for region: 059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:21,435 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/b9608c129d224c4fa22a6083bbb53907 for region: 059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:21,435 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/b3d0ee38ca69423f9a6877578518bc76 for region: 059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:21,444 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/0837374a7f2a4c82bfa97443781ee242 for region: 059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:21,444 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/21a49995122e48a493b6d0a8abb3ff52 for region: 059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:21,444 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/84b2009af5f3401287c117850ef9fc4b for region: 059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:21,448 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/8f67930f70934c80893ea92e3da97da1 for region: 059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:21,448 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/b1149ff9cf704fc0b06d4a93d73229dc for region: 059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:21,449 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/bc695fd948ff4e53b03206a746904fa3 for region: 059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:21,449 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/e5774ef6b56443f583eeef35a0086dd7 for region: 059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:21,631 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/902bebca210f45908626f299bace6411 for region: 059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:21,639 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/4ddd3ec587eb4d9f9c2d89d53cdc963f for region: 059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:21,652 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/b8675e8553444d2bbde903dca91aba1e for region: 059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:21,666 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/b9608c129d224c4fa22a6083bbb53907 for region: 059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:21,684 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/b1149ff9cf704fc0b06d4a93d73229dc for region: 059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:21,690 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-05-02 02:55:21,704 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/e5774ef6b56443f583eeef35a0086dd7 for region: 059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:21,709 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/b3d0ee38ca69423f9a6877578518bc76 for region: 059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:21,709 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/8f67930f70934c80893ea92e3da97da1 for region: 059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:21,711 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/bc695fd948ff4e53b03206a746904fa3 for region: 059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:21,712 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/0837374a7f2a4c82bfa97443781ee242 for region: 059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:22,017 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/9a1be768ec4e4cdbadc55ffdbfe82828 for region: 059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:22,023 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/84b2009af5f3401287c117850ef9fc4b for region: 059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:22,024 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/78391fd40923430ba069adcd774c9f4b for region: 059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:22,111 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/21a49995122e48a493b6d0a8abb3ff52 for region: 059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:22,111 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region 059f75e2e092770bf7a63b32739cb003 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-05-02 02:55:22,541 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-02 02:55:22,541 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:22,541 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-02 02:55:22,541 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-02 02:55:22,670 INFO [Listener at localhost/37119] procedure.MasterProcedureTestingUtility(454): Restart 4 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:22,670 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-02 02:55:22,670 INFO [Listener at localhost/37119] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-02 02:55:22,670 INFO [Listener at localhost/37119] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-02 02:55:22,670 ERROR [Listener at localhost/37119] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-02 02:55:22,670 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-02 02:55:22,670 INFO [Listener at localhost/37119] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-02 02:55:22,670 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-02 02:55:22,670 INFO [Listener at localhost/37119] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-02 02:55:22,671 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-02 02:55:22,673 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-05-02 02:55:22,673 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-02 02:55:22,673 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-02 02:55:22,673 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-02 02:55:22,673 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-02 02:55:22,674 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-02 02:55:22,674 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-02 02:55:22,674 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:55:22,674 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=084b1607f7ed8615120a04e4729fbafa, daughterA=ea7683cd3436a62eb16418a3c875d563, daughterB=408e891cab9b919f4038b233ba34779d 2023-05-02 02:55:22,675 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:55:22,675 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:55:22,675 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-02 02:55:22,675 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:55:22,675 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-02 02:55:22,676 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-02 02:55:22,676 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-02 02:55:22,676 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:55:22,676 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, ASSIGN 2023-05-02 02:55:22,676 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-05-02 02:55:22,676 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-05-02 02:55:22,677 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-02 02:55:22,677 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=afa57eddfa2d26bf35b12dc022351e78, daughterA=b91a201af734ea9cb01587589c96e908, daughterB=292a312492e2395ec0ca83a160894be9 2023-05-02 02:55:22,677 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-02 02:55:22,677 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-02 02:55:22,677 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:55:22,678 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=8af7d4111b21c70efea08f7beab73a9f, daughterA=bb895e1df53cda46bfcb5f94fe1710c9, daughterB=024f19513d3a9ded6c060054f1607e68 2023-05-02 02:55:22,678 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:55:22,678 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:55:22,678 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-02 02:55:22,678 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-02 02:55:22,679 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,39081,1682996021579, table=testRollbackForSplitTableRegionWithReplica, region=1b91efe506b89bc57ba68287b021be4f is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=74b19512a655e6f00fc418d81fba16c8, daughterA=bfa227908c6548a4187e4deb87e808a6, daughterB=da50a5f8e634bd92f99eebe500206397 2023-05-02 02:55:22,679 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-02 02:55:22,679 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-02 02:55:22,679 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-05-02 02:55:22,679 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-05-02 02:55:22,679 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-05-02 02:55:22,680 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-05-02 02:55:22,680 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:22,680 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=059f75e2e092770bf7a63b32739cb003, UNASSIGN 2023-05-02 02:55:22,680 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 059f75e2e092770bf7a63b32739cb003, server=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:22,680 DEBUG [Listener at localhost/37119] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 held the lock before restarting, call acquireLock to restore it. 2023-05-02 02:55:22,680 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:22,680 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:22,680 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:22,680 DEBUG [Listener at localhost/37119] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=059f75e2e092770bf7a63b32739cb003, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-02 02:55:22,680 DEBUG [Listener at localhost/37119] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 059f75e2e092770bf7a63b32739cb003, server=jenkins-hbase4.apache.org,42019,1682996021676 didn't hold the lock before restarting, skip acquiring lock. 2023-05-02 02:55:22,681 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-05-02 02:55:22,681 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-02 02:55:22,682 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,38925,1682996021633, table=hbase:meta, region=1588230740 2023-05-02 02:55:22,688 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682996152688 2023-05-02 02:55:22,689 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(1517): Joining cluster... 2023-05-02 02:55:22,689 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:22,692 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=5c823d6f15ad7b8fe82f9ae4beee6513, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42019,1682996021676, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676, openSeqNum=2 2023-05-02 02:55:22,692 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=059f75e2e092770bf7a63b32739cb003, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,42019,1682996021676, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676, openSeqNum=2 2023-05-02 02:55:22,692 INFO [Listener at localhost/37119] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-02 02:55:22,692 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682996182692 2023-05-02 02:55:22,692 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682996242692 2023-05-02 02:55:22,692 INFO [Listener at localhost/37119] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-05-02 02:55:22,700 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=059f75e2e092770bf7a63b32739cb003, threads=14 2023-05-02 02:55:22,701 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/4ddd3ec587eb4d9f9c2d89d53cdc963f for region: 059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:22,701 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/78391fd40923430ba069adcd774c9f4b for region: 059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:22,701 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/902bebca210f45908626f299bace6411 for region: 059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:22,701 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/9a1be768ec4e4cdbadc55ffdbfe82828 for region: 059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:22,702 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/b3d0ee38ca69423f9a6877578518bc76 for region: 059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:22,709 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/b8675e8553444d2bbde903dca91aba1e for region: 059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:22,710 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/b9608c129d224c4fa22a6083bbb53907 for region: 059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:22,710 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/0837374a7f2a4c82bfa97443781ee242 for region: 059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:22,710 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/84b2009af5f3401287c117850ef9fc4b for region: 059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:22,710 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/21a49995122e48a493b6d0a8abb3ff52 for region: 059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:22,710 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/8f67930f70934c80893ea92e3da97da1 for region: 059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:22,710 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/b1149ff9cf704fc0b06d4a93d73229dc for region: 059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:22,710 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/bc695fd948ff4e53b03206a746904fa3 for region: 059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:22,710 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/e5774ef6b56443f583eeef35a0086dd7 for region: 059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:22,827 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/78391fd40923430ba069adcd774c9f4b for region: 059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:22,844 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/bc695fd948ff4e53b03206a746904fa3 for region: 059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:22,856 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/9a1be768ec4e4cdbadc55ffdbfe82828 for region: 059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:22,856 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/4ddd3ec587eb4d9f9c2d89d53cdc963f for region: 059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:22,868 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/21a49995122e48a493b6d0a8abb3ff52 for region: 059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:22,873 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/b3d0ee38ca69423f9a6877578518bc76 for region: 059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:22,882 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/902bebca210f45908626f299bace6411 for region: 059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:22,882 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/8f67930f70934c80893ea92e3da97da1 for region: 059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:22,882 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/84b2009af5f3401287c117850ef9fc4b for region: 059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:22,886 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/0837374a7f2a4c82bfa97443781ee242 for region: 059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:22,887 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/b1149ff9cf704fc0b06d4a93d73229dc for region: 059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:22,887 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/b8675e8553444d2bbde903dca91aba1e for region: 059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:22,888 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/b9608c129d224c4fa22a6083bbb53907 for region: 059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:23,225 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/e5774ef6b56443f583eeef35a0086dd7 for region: 059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:23,225 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region 059f75e2e092770bf7a63b32739cb003 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-05-02 02:55:23,257 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-02 02:55:23,262 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-05-02 02:55:23,264 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-05-02 02:55:23,264 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-02 02:55:23,264 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:23,264 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-02 02:55:23,264 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-02 02:55:23,442 INFO [Listener at localhost/37119] procedure.MasterProcedureTestingUtility(454): Restart 5 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:23,443 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-02 02:55:23,443 INFO [Listener at localhost/37119] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-02 02:55:23,443 INFO [Listener at localhost/37119] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-02 02:55:23,443 ERROR [Listener at localhost/37119] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-02 02:55:23,443 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-02 02:55:23,443 INFO [Listener at localhost/37119] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-02 02:55:23,443 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-02 02:55:23,443 INFO [Listener at localhost/37119] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-02 02:55:23,444 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-02 02:55:23,445 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-05-02 02:55:23,445 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-02 02:55:23,445 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-02 02:55:23,445 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-02 02:55:23,445 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-02 02:55:23,446 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-02 02:55:23,446 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-02 02:55:23,446 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:55:23,446 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=084b1607f7ed8615120a04e4729fbafa, daughterA=ea7683cd3436a62eb16418a3c875d563, daughterB=408e891cab9b919f4038b233ba34779d 2023-05-02 02:55:23,446 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:55:23,446 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:55:23,446 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-02 02:55:23,447 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:55:23,447 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-02 02:55:23,447 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-02 02:55:23,447 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-02 02:55:23,447 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:55:23,447 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, ASSIGN 2023-05-02 02:55:23,447 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-05-02 02:55:23,447 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-05-02 02:55:23,447 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-02 02:55:23,448 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=afa57eddfa2d26bf35b12dc022351e78, daughterA=b91a201af734ea9cb01587589c96e908, daughterB=292a312492e2395ec0ca83a160894be9 2023-05-02 02:55:23,448 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-02 02:55:23,448 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-02 02:55:23,448 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:55:23,448 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=8af7d4111b21c70efea08f7beab73a9f, daughterA=bb895e1df53cda46bfcb5f94fe1710c9, daughterB=024f19513d3a9ded6c060054f1607e68 2023-05-02 02:55:23,448 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:55:23,448 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:55:23,448 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-02 02:55:23,449 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-02 02:55:23,449 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,39081,1682996021579, table=testRollbackForSplitTableRegionWithReplica, region=1b91efe506b89bc57ba68287b021be4f is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=74b19512a655e6f00fc418d81fba16c8, daughterA=bfa227908c6548a4187e4deb87e808a6, daughterB=da50a5f8e634bd92f99eebe500206397 2023-05-02 02:55:23,449 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-02 02:55:23,449 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-02 02:55:23,449 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-05-02 02:55:23,449 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-05-02 02:55:23,449 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-05-02 02:55:23,449 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-05-02 02:55:23,450 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:23,450 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=059f75e2e092770bf7a63b32739cb003, UNASSIGN 2023-05-02 02:55:23,450 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 059f75e2e092770bf7a63b32739cb003, server=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:23,450 DEBUG [Listener at localhost/37119] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 held the lock before restarting, call acquireLock to restore it. 2023-05-02 02:55:23,450 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:23,450 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:23,450 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:23,450 DEBUG [Listener at localhost/37119] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=059f75e2e092770bf7a63b32739cb003, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-02 02:55:23,450 DEBUG [Listener at localhost/37119] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 059f75e2e092770bf7a63b32739cb003, server=jenkins-hbase4.apache.org,42019,1682996021676 didn't hold the lock before restarting, skip acquiring lock. 2023-05-02 02:55:23,450 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-05-02 02:55:23,451 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-02 02:55:23,451 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,38925,1682996021633, table=hbase:meta, region=1588230740 2023-05-02 02:55:23,452 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682996153452 2023-05-02 02:55:23,453 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:23,453 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(1517): Joining cluster... 2023-05-02 02:55:23,456 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=5c823d6f15ad7b8fe82f9ae4beee6513, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42019,1682996021676, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676, openSeqNum=2 2023-05-02 02:55:23,456 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=059f75e2e092770bf7a63b32739cb003, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,42019,1682996021676, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676, openSeqNum=2 2023-05-02 02:55:23,456 INFO [Listener at localhost/37119] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-02 02:55:23,456 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682996183456 2023-05-02 02:55:23,456 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682996243456 2023-05-02 02:55:23,456 INFO [Listener at localhost/37119] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-02 02:55:23,459 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-02 02:55:23,461 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-02 02:55:23,461 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:23,461 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-02 02:55:23,461 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-02 02:55:23,707 INFO [Listener at localhost/37119] procedure.MasterProcedureTestingUtility(454): Restart 6 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:23,707 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-02 02:55:23,707 INFO [Listener at localhost/37119] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-02 02:55:23,707 INFO [Listener at localhost/37119] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-02 02:55:23,707 ERROR [Listener at localhost/37119] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-02 02:55:23,707 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-02 02:55:23,707 INFO [Listener at localhost/37119] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-02 02:55:23,707 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-02 02:55:23,707 INFO [Listener at localhost/37119] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-02 02:55:23,708 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-02 02:55:23,709 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-05-02 02:55:23,709 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-02 02:55:23,709 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-02 02:55:23,710 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-02 02:55:23,710 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-02 02:55:23,710 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-02 02:55:23,710 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-02 02:55:23,710 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:55:23,710 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=084b1607f7ed8615120a04e4729fbafa, daughterA=ea7683cd3436a62eb16418a3c875d563, daughterB=408e891cab9b919f4038b233ba34779d 2023-05-02 02:55:23,710 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:55:23,711 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:55:23,711 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-02 02:55:23,711 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:55:23,711 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-02 02:55:23,711 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-02 02:55:23,711 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-02 02:55:23,711 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:55:23,711 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, ASSIGN 2023-05-02 02:55:23,712 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-05-02 02:55:23,712 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-05-02 02:55:23,712 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-02 02:55:23,712 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=afa57eddfa2d26bf35b12dc022351e78, daughterA=b91a201af734ea9cb01587589c96e908, daughterB=292a312492e2395ec0ca83a160894be9 2023-05-02 02:55:23,712 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-02 02:55:23,712 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-02 02:55:23,712 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:55:23,713 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=8af7d4111b21c70efea08f7beab73a9f, daughterA=bb895e1df53cda46bfcb5f94fe1710c9, daughterB=024f19513d3a9ded6c060054f1607e68 2023-05-02 02:55:23,713 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:55:23,713 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:55:23,713 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-02 02:55:23,713 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-02 02:55:23,714 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,39081,1682996021579, table=testRollbackForSplitTableRegionWithReplica, region=1b91efe506b89bc57ba68287b021be4f is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=74b19512a655e6f00fc418d81fba16c8, daughterA=bfa227908c6548a4187e4deb87e808a6, daughterB=da50a5f8e634bd92f99eebe500206397 2023-05-02 02:55:23,714 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-02 02:55:23,714 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-02 02:55:23,714 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-05-02 02:55:23,714 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-05-02 02:55:23,714 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-05-02 02:55:23,714 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-05-02 02:55:23,715 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:23,715 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=059f75e2e092770bf7a63b32739cb003, UNASSIGN 2023-05-02 02:55:23,715 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 059f75e2e092770bf7a63b32739cb003, server=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:23,715 DEBUG [Listener at localhost/37119] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 held the lock before restarting, call acquireLock to restore it. 2023-05-02 02:55:23,715 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:23,715 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:23,715 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:23,715 DEBUG [Listener at localhost/37119] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=059f75e2e092770bf7a63b32739cb003, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-02 02:55:23,715 DEBUG [Listener at localhost/37119] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 059f75e2e092770bf7a63b32739cb003, server=jenkins-hbase4.apache.org,42019,1682996021676 didn't hold the lock before restarting, skip acquiring lock. 2023-05-02 02:55:23,715 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-05-02 02:55:23,716 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-02 02:55:23,716 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,38925,1682996021633, table=hbase:meta, region=1588230740 2023-05-02 02:55:23,720 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682996153720 2023-05-02 02:55:23,721 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(1517): Joining cluster... 2023-05-02 02:55:23,721 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:23,726 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=5c823d6f15ad7b8fe82f9ae4beee6513, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42019,1682996021676, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676, openSeqNum=2 2023-05-02 02:55:23,726 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=059f75e2e092770bf7a63b32739cb003, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,42019,1682996021676, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676, openSeqNum=2 2023-05-02 02:55:23,726 INFO [Listener at localhost/37119] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-02 02:55:23,726 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682996183726 2023-05-02 02:55:23,726 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682996243726 2023-05-02 02:55:23,726 INFO [Listener at localhost/37119] assignment.AssignmentManager(1536): Joined the cluster in 5 msec 2023-05-02 02:55:23,726 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-02 02:55:23,729 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1682996118490.059f75e2e092770bf7a63b32739cb003.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1682996123728"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1682996123728"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1682996123728"}]},"ts":"1682996123728"} 2023-05-02 02:55:23,729 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1682996120349.78fb849dbb85ed1018811c0b9ad29f02.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1682996123728"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682996123728"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682996123728"}]},"ts":"1682996123728"} 2023-05-02 02:55:23,729 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1682996120349.f920d56a4f8ae9570632b12801c0f779.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1682996123728"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682996123728"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682996123728"}]},"ts":"1682996123728"} 2023-05-02 02:55:23,730 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-02 02:55:23,730 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:23,730 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-02 02:55:23,730 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-02 02:55:23,977 INFO [Listener at localhost/37119] procedure.MasterProcedureTestingUtility(454): Restart 7 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:23,977 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-02 02:55:23,977 INFO [Listener at localhost/37119] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-02 02:55:23,977 INFO [Listener at localhost/37119] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-02 02:55:23,977 ERROR [Listener at localhost/37119] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-02 02:55:23,977 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-02 02:55:23,977 INFO [Listener at localhost/37119] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-02 02:55:23,977 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-02 02:55:23,977 INFO [Listener at localhost/37119] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-02 02:55:23,978 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-02 02:55:23,979 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-05-02 02:55:23,979 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-02 02:55:23,979 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-02 02:55:23,979 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-02 02:55:23,980 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-02 02:55:23,980 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-02 02:55:23,980 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-02 02:55:23,980 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:55:23,980 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=084b1607f7ed8615120a04e4729fbafa, daughterA=ea7683cd3436a62eb16418a3c875d563, daughterB=408e891cab9b919f4038b233ba34779d 2023-05-02 02:55:23,980 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:55:23,980 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:55:23,981 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-02 02:55:23,981 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:55:23,981 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-02 02:55:23,981 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-02 02:55:23,982 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-02 02:55:23,982 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:55:23,982 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, ASSIGN 2023-05-02 02:55:23,983 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-05-02 02:55:23,983 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-05-02 02:55:23,983 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-02 02:55:23,983 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=afa57eddfa2d26bf35b12dc022351e78, daughterA=b91a201af734ea9cb01587589c96e908, daughterB=292a312492e2395ec0ca83a160894be9 2023-05-02 02:55:23,983 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-02 02:55:23,984 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-02 02:55:23,984 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:55:23,984 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=8af7d4111b21c70efea08f7beab73a9f, daughterA=bb895e1df53cda46bfcb5f94fe1710c9, daughterB=024f19513d3a9ded6c060054f1607e68 2023-05-02 02:55:23,984 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:55:23,984 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:55:23,984 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-02 02:55:23,985 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-02 02:55:23,985 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,39081,1682996021579, table=testRollbackForSplitTableRegionWithReplica, region=1b91efe506b89bc57ba68287b021be4f is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=74b19512a655e6f00fc418d81fba16c8, daughterA=bfa227908c6548a4187e4deb87e808a6, daughterB=da50a5f8e634bd92f99eebe500206397 2023-05-02 02:55:23,985 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-02 02:55:23,985 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-02 02:55:23,985 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-05-02 02:55:23,985 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-05-02 02:55:23,986 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-05-02 02:55:23,986 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-05-02 02:55:23,986 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:23,986 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=059f75e2e092770bf7a63b32739cb003, UNASSIGN 2023-05-02 02:55:23,986 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 059f75e2e092770bf7a63b32739cb003, server=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:23,986 DEBUG [Listener at localhost/37119] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 held the lock before restarting, call acquireLock to restore it. 2023-05-02 02:55:23,986 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:23,986 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:23,986 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:23,986 DEBUG [Listener at localhost/37119] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=059f75e2e092770bf7a63b32739cb003, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-02 02:55:23,987 DEBUG [Listener at localhost/37119] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 059f75e2e092770bf7a63b32739cb003, server=jenkins-hbase4.apache.org,42019,1682996021676 didn't hold the lock before restarting, skip acquiring lock. 2023-05-02 02:55:23,987 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-05-02 02:55:23,987 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-02 02:55:23,988 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,38925,1682996021633, table=hbase:meta, region=1588230740 2023-05-02 02:55:23,988 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682996153988 2023-05-02 02:55:23,990 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:23,990 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(1517): Joining cluster... 2023-05-02 02:55:23,992 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=5c823d6f15ad7b8fe82f9ae4beee6513, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42019,1682996021676, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676, openSeqNum=2 2023-05-02 02:55:23,993 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=059f75e2e092770bf7a63b32739cb003, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,42019,1682996021676, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676, openSeqNum=2 2023-05-02 02:55:23,993 INFO [Listener at localhost/37119] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,42019,1682996021676, table=testRecoveryAndDoubleExecution, region=059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:23,993 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=78fb849dbb85ed1018811c0b9ad29f02, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-05-02 02:55:23,993 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=f920d56a4f8ae9570632b12801c0f779, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-05-02 02:55:23,994 INFO [Listener at localhost/37119] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-02 02:55:23,994 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682996183994 2023-05-02 02:55:23,994 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682996243994 2023-05-02 02:55:23,994 INFO [Listener at localhost/37119] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-02 02:55:23,994 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1682996118490.059f75e2e092770bf7a63b32739cb003.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1682996123994"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1682996123994"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1682996123994"}]},"ts":"1682996123994"} 2023-05-02 02:55:23,994 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1682996120349.78fb849dbb85ed1018811c0b9ad29f02.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1682996123994"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682996123994"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682996123994"}]},"ts":"1682996123994"} 2023-05-02 02:55:23,994 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1682996120349.f920d56a4f8ae9570632b12801c0f779.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1682996123994"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682996123994"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682996123994"}]},"ts":"1682996123994"} 2023-05-02 02:55:23,996 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-02 02:55:23,998 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-02 02:55:23,998 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:23,998 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-02 02:55:23,998 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-02 02:55:24,213 ERROR [BP-376276558-172.31.14.131-1682996016523 heartbeating to localhost/127.0.0.1:38649] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-376276558-172.31.14.131-1682996016523 (Datanode Uuid e5b41dfb-e591-48a4-b02d-848c5b0dba36) service to localhost/127.0.0.1:38649 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-05-02 02:55:24,244 INFO [Listener at localhost/37119] procedure.MasterProcedureTestingUtility(454): Restart 8 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:24,244 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-02 02:55:24,244 INFO [Listener at localhost/37119] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-02 02:55:24,244 INFO [Listener at localhost/37119] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-02 02:55:24,245 ERROR [Listener at localhost/37119] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-02 02:55:24,245 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-02 02:55:24,245 INFO [Listener at localhost/37119] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-02 02:55:24,245 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-02 02:55:24,245 INFO [Listener at localhost/37119] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-02 02:55:24,246 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-02 02:55:24,246 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11] 2023-05-02 02:55:24,247 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-02 02:55:24,247 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-02 02:55:24,247 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-02 02:55:24,247 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-02 02:55:24,247 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-02 02:55:24,247 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-02 02:55:24,248 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:55:24,248 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=084b1607f7ed8615120a04e4729fbafa, daughterA=ea7683cd3436a62eb16418a3c875d563, daughterB=408e891cab9b919f4038b233ba34779d 2023-05-02 02:55:24,248 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:55:24,248 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:55:24,248 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-02 02:55:24,248 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:55:24,248 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-02 02:55:24,248 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-02 02:55:24,249 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-02 02:55:24,249 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:55:24,249 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, ASSIGN 2023-05-02 02:55:24,249 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-05-02 02:55:24,249 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-05-02 02:55:24,249 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-02 02:55:24,249 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=afa57eddfa2d26bf35b12dc022351e78, daughterA=b91a201af734ea9cb01587589c96e908, daughterB=292a312492e2395ec0ca83a160894be9 2023-05-02 02:55:24,249 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-02 02:55:24,250 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-02 02:55:24,250 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:55:24,250 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=8af7d4111b21c70efea08f7beab73a9f, daughterA=bb895e1df53cda46bfcb5f94fe1710c9, daughterB=024f19513d3a9ded6c060054f1607e68 2023-05-02 02:55:24,250 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:55:24,250 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:55:24,250 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-02 02:55:24,250 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-02 02:55:24,251 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,39081,1682996021579, table=testRollbackForSplitTableRegionWithReplica, region=1b91efe506b89bc57ba68287b021be4f is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=74b19512a655e6f00fc418d81fba16c8, daughterA=bfa227908c6548a4187e4deb87e808a6, daughterB=da50a5f8e634bd92f99eebe500206397 2023-05-02 02:55:24,251 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-02 02:55:24,251 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-02 02:55:24,251 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-05-02 02:55:24,251 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-05-02 02:55:24,251 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-05-02 02:55:24,251 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-05-02 02:55:24,251 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:24,252 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=059f75e2e092770bf7a63b32739cb003, UNASSIGN 2023-05-02 02:55:24,252 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 059f75e2e092770bf7a63b32739cb003, server=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:24,252 DEBUG [Listener at localhost/37119] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 held the lock before restarting, call acquireLock to restore it. 2023-05-02 02:55:24,252 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:24,252 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:24,252 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:24,252 DEBUG [Listener at localhost/37119] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=059f75e2e092770bf7a63b32739cb003, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-02 02:55:24,252 DEBUG [Listener at localhost/37119] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 059f75e2e092770bf7a63b32739cb003, server=jenkins-hbase4.apache.org,42019,1682996021676 didn't hold the lock before restarting, skip acquiring lock. 2023-05-02 02:55:24,252 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-05-02 02:55:24,253 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-02 02:55:24,253 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,38925,1682996021633, table=hbase:meta, region=1588230740 2023-05-02 02:55:24,253 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682996154253 2023-05-02 02:55:24,253 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(1517): Joining cluster... 2023-05-02 02:55:24,255 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:24,258 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=5c823d6f15ad7b8fe82f9ae4beee6513, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42019,1682996021676, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676, openSeqNum=2 2023-05-02 02:55:24,258 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=059f75e2e092770bf7a63b32739cb003, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,42019,1682996021676, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676, openSeqNum=2 2023-05-02 02:55:24,258 INFO [Listener at localhost/37119] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,42019,1682996021676, table=testRecoveryAndDoubleExecution, region=059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:24,258 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=78fb849dbb85ed1018811c0b9ad29f02, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-05-02 02:55:24,258 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=f920d56a4f8ae9570632b12801c0f779, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-05-02 02:55:24,258 INFO [Listener at localhost/37119] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-02 02:55:24,258 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682996184258 2023-05-02 02:55:24,258 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682996244258 2023-05-02 02:55:24,258 INFO [Listener at localhost/37119] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-05-02 02:55:24,258 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-02 02:55:24,260 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=78fb849dbb85ed1018811c0b9ad29f02, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=f920d56a4f8ae9570632b12801c0f779, ASSIGN}] 2023-05-02 02:55:24,261 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-02 02:55:24,261 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:24,261 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-02 02:55:24,261 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-02 02:55:24,508 INFO [Listener at localhost/37119] procedure.MasterProcedureTestingUtility(454): Restart 9 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:24,509 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-02 02:55:24,509 INFO [Listener at localhost/37119] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-02 02:55:24,509 INFO [Listener at localhost/37119] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-02 02:55:24,509 ERROR [Listener at localhost/37119] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-02 02:55:24,509 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-02 02:55:24,509 INFO [Listener at localhost/37119] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-02 02:55:24,509 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-02 02:55:24,509 INFO [Listener at localhost/37119] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-02 02:55:24,510 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-02 02:55:24,511 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12] 2023-05-02 02:55:24,511 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-02 02:55:24,511 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-02 02:55:24,511 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-02 02:55:24,512 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-02 02:55:24,512 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-02 02:55:24,512 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-02 02:55:24,512 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:55:24,512 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=084b1607f7ed8615120a04e4729fbafa, daughterA=ea7683cd3436a62eb16418a3c875d563, daughterB=408e891cab9b919f4038b233ba34779d 2023-05-02 02:55:24,512 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:55:24,512 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:55:24,513 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-02 02:55:24,513 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:55:24,513 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-02 02:55:24,513 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-02 02:55:24,513 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-02 02:55:24,513 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:55:24,513 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, ASSIGN 2023-05-02 02:55:24,514 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-05-02 02:55:24,514 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-05-02 02:55:24,514 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-02 02:55:24,514 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=afa57eddfa2d26bf35b12dc022351e78, daughterA=b91a201af734ea9cb01587589c96e908, daughterB=292a312492e2395ec0ca83a160894be9 2023-05-02 02:55:24,514 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-02 02:55:24,514 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-02 02:55:24,514 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:55:24,515 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=8af7d4111b21c70efea08f7beab73a9f, daughterA=bb895e1df53cda46bfcb5f94fe1710c9, daughterB=024f19513d3a9ded6c060054f1607e68 2023-05-02 02:55:24,515 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:55:24,515 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:55:24,515 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-02 02:55:24,515 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-02 02:55:24,515 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,39081,1682996021579, table=testRollbackForSplitTableRegionWithReplica, region=1b91efe506b89bc57ba68287b021be4f is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=74b19512a655e6f00fc418d81fba16c8, daughterA=bfa227908c6548a4187e4deb87e808a6, daughterB=da50a5f8e634bd92f99eebe500206397 2023-05-02 02:55:24,515 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-02 02:55:24,516 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-02 02:55:24,516 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-05-02 02:55:24,516 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-05-02 02:55:24,516 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-05-02 02:55:24,516 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-05-02 02:55:24,516 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:24,516 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=059f75e2e092770bf7a63b32739cb003, UNASSIGN 2023-05-02 02:55:24,516 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 059f75e2e092770bf7a63b32739cb003, server=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:24,516 DEBUG [Listener at localhost/37119] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 held the lock before restarting, call acquireLock to restore it. 2023-05-02 02:55:24,517 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:24,517 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:24,517 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:24,517 DEBUG [Listener at localhost/37119] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=059f75e2e092770bf7a63b32739cb003, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-02 02:55:24,517 DEBUG [Listener at localhost/37119] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 059f75e2e092770bf7a63b32739cb003, server=jenkins-hbase4.apache.org,42019,1682996021676 didn't hold the lock before restarting, skip acquiring lock. 2023-05-02 02:55:24,517 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-05-02 02:55:24,517 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-02 02:55:24,518 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,38925,1682996021633, table=hbase:meta, region=1588230740 2023-05-02 02:55:24,518 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682996154518 2023-05-02 02:55:24,520 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:24,520 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(1517): Joining cluster... 2023-05-02 02:55:24,522 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=5c823d6f15ad7b8fe82f9ae4beee6513, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42019,1682996021676, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676, openSeqNum=2 2023-05-02 02:55:24,522 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=059f75e2e092770bf7a63b32739cb003, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,42019,1682996021676, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676, openSeqNum=2 2023-05-02 02:55:24,522 INFO [Listener at localhost/37119] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,42019,1682996021676, table=testRecoveryAndDoubleExecution, region=059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:24,522 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=78fb849dbb85ed1018811c0b9ad29f02, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-05-02 02:55:24,522 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=f920d56a4f8ae9570632b12801c0f779, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-05-02 02:55:24,522 INFO [Listener at localhost/37119] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-02 02:55:24,522 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682996184522 2023-05-02 02:55:24,522 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682996244522 2023-05-02 02:55:24,522 INFO [Listener at localhost/37119] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-05-02 02:55:24,523 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=78fb849dbb85ed1018811c0b9ad29f02, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=f920d56a4f8ae9570632b12801c0f779, ASSIGN}] 2023-05-02 02:55:24,523 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-02 02:55:24,525 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=f920d56a4f8ae9570632b12801c0f779, ASSIGN 2023-05-02 02:55:24,525 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=f920d56a4f8ae9570632b12801c0f779, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,42019,1682996021676; forceNewPlan=false, retain=false 2023-05-02 02:55:24,526 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=78fb849dbb85ed1018811c0b9ad29f02, ASSIGN 2023-05-02 02:55:24,526 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=78fb849dbb85ed1018811c0b9ad29f02, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,42019,1682996021676; forceNewPlan=false, retain=false 2023-05-02 02:55:24,675 INFO [jenkins-hbase4:41119] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-05-02 02:55:24,677 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=78fb849dbb85ed1018811c0b9ad29f02, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:24,677 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1682996120349.78fb849dbb85ed1018811c0b9ad29f02.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1682996124677"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1682996124677"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682996124677"}]},"ts":"1682996124677"} 2023-05-02 02:55:24,679 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=117, ppid=115, state=RUNNABLE; OpenRegionProcedure 78fb849dbb85ed1018811c0b9ad29f02, server=jenkins-hbase4.apache.org,42019,1682996021676}] 2023-05-02 02:55:24,681 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=f920d56a4f8ae9570632b12801c0f779, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:24,681 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1682996120349.f920d56a4f8ae9570632b12801c0f779.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1682996124681"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1682996124681"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682996124681"}]},"ts":"1682996124681"} 2023-05-02 02:55:24,682 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=118, ppid=116, state=RUNNABLE; OpenRegionProcedure f920d56a4f8ae9570632b12801c0f779, server=jenkins-hbase4.apache.org,42019,1682996021676}] 2023-05-02 02:55:24,832 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRecoveryAndDoubleExecution,,1682996120349.78fb849dbb85ed1018811c0b9ad29f02. 2023-05-02 02:55:24,833 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 78fb849dbb85ed1018811c0b9ad29f02, NAME => 'testRecoveryAndDoubleExecution,,1682996120349.78fb849dbb85ed1018811c0b9ad29f02.', STARTKEY => '', ENDKEY => '41'} 2023-05-02 02:55:24,833 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-02 02:55:24,833 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 78fb849dbb85ed1018811c0b9ad29f02 2023-05-02 02:55:24,833 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRecoveryAndDoubleExecution,,1682996120349.78fb849dbb85ed1018811c0b9ad29f02.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-02 02:55:24,833 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 78fb849dbb85ed1018811c0b9ad29f02 2023-05-02 02:55:24,833 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 78fb849dbb85ed1018811c0b9ad29f02 2023-05-02 02:55:24,834 INFO [StoreOpener-78fb849dbb85ed1018811c0b9ad29f02-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 78fb849dbb85ed1018811c0b9ad29f02 2023-05-02 02:55:24,835 DEBUG [StoreOpener-78fb849dbb85ed1018811c0b9ad29f02-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf1 2023-05-02 02:55:24,835 DEBUG [StoreOpener-78fb849dbb85ed1018811c0b9ad29f02-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf1 2023-05-02 02:55:24,836 INFO [StoreOpener-78fb849dbb85ed1018811c0b9ad29f02-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 78fb849dbb85ed1018811c0b9ad29f02 columnFamilyName cf1 2023-05-02 02:55:24,847 DEBUG [StoreOpener-78fb849dbb85ed1018811c0b9ad29f02-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf1/0837374a7f2a4c82bfa97443781ee242.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/0837374a7f2a4c82bfa97443781ee242-bottom 2023-05-02 02:55:24,850 DEBUG [StoreOpener-78fb849dbb85ed1018811c0b9ad29f02-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf1/21a49995122e48a493b6d0a8abb3ff52.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/21a49995122e48a493b6d0a8abb3ff52-bottom 2023-05-02 02:55:24,854 DEBUG [StoreOpener-78fb849dbb85ed1018811c0b9ad29f02-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf1/84b2009af5f3401287c117850ef9fc4b.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/84b2009af5f3401287c117850ef9fc4b-bottom 2023-05-02 02:55:24,858 DEBUG [StoreOpener-78fb849dbb85ed1018811c0b9ad29f02-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf1/8f67930f70934c80893ea92e3da97da1.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/8f67930f70934c80893ea92e3da97da1-bottom 2023-05-02 02:55:24,861 DEBUG [StoreOpener-78fb849dbb85ed1018811c0b9ad29f02-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf1/b1149ff9cf704fc0b06d4a93d73229dc.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/b1149ff9cf704fc0b06d4a93d73229dc-bottom 2023-05-02 02:55:24,865 DEBUG [StoreOpener-78fb849dbb85ed1018811c0b9ad29f02-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf1/bc695fd948ff4e53b03206a746904fa3.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/bc695fd948ff4e53b03206a746904fa3-bottom 2023-05-02 02:55:24,872 DEBUG [StoreOpener-78fb849dbb85ed1018811c0b9ad29f02-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf1/e5774ef6b56443f583eeef35a0086dd7.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/e5774ef6b56443f583eeef35a0086dd7-bottom 2023-05-02 02:55:24,872 INFO [StoreOpener-78fb849dbb85ed1018811c0b9ad29f02-1] regionserver.HStore(310): Store=78fb849dbb85ed1018811c0b9ad29f02/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-02 02:55:24,872 INFO [StoreOpener-78fb849dbb85ed1018811c0b9ad29f02-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 78fb849dbb85ed1018811c0b9ad29f02 2023-05-02 02:55:24,873 DEBUG [StoreOpener-78fb849dbb85ed1018811c0b9ad29f02-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf2 2023-05-02 02:55:24,873 DEBUG [StoreOpener-78fb849dbb85ed1018811c0b9ad29f02-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf2 2023-05-02 02:55:24,873 INFO [StoreOpener-78fb849dbb85ed1018811c0b9ad29f02-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 78fb849dbb85ed1018811c0b9ad29f02 columnFamilyName cf2 2023-05-02 02:55:24,885 DEBUG [StoreOpener-78fb849dbb85ed1018811c0b9ad29f02-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf2/4ddd3ec587eb4d9f9c2d89d53cdc963f.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/4ddd3ec587eb4d9f9c2d89d53cdc963f-bottom 2023-05-02 02:55:24,888 DEBUG [StoreOpener-78fb849dbb85ed1018811c0b9ad29f02-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf2/78391fd40923430ba069adcd774c9f4b.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/78391fd40923430ba069adcd774c9f4b-bottom 2023-05-02 02:55:24,892 DEBUG [StoreOpener-78fb849dbb85ed1018811c0b9ad29f02-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf2/902bebca210f45908626f299bace6411.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/902bebca210f45908626f299bace6411-bottom 2023-05-02 02:55:24,896 DEBUG [StoreOpener-78fb849dbb85ed1018811c0b9ad29f02-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf2/9a1be768ec4e4cdbadc55ffdbfe82828.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/9a1be768ec4e4cdbadc55ffdbfe82828-bottom 2023-05-02 02:55:24,899 DEBUG [StoreOpener-78fb849dbb85ed1018811c0b9ad29f02-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf2/b3d0ee38ca69423f9a6877578518bc76.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/b3d0ee38ca69423f9a6877578518bc76-bottom 2023-05-02 02:55:24,902 DEBUG [StoreOpener-78fb849dbb85ed1018811c0b9ad29f02-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf2/b8675e8553444d2bbde903dca91aba1e.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/b8675e8553444d2bbde903dca91aba1e-bottom 2023-05-02 02:55:24,906 DEBUG [StoreOpener-78fb849dbb85ed1018811c0b9ad29f02-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf2/b9608c129d224c4fa22a6083bbb53907.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/b9608c129d224c4fa22a6083bbb53907-bottom 2023-05-02 02:55:24,906 INFO [StoreOpener-78fb849dbb85ed1018811c0b9ad29f02-1] regionserver.HStore(310): Store=78fb849dbb85ed1018811c0b9ad29f02/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-02 02:55:24,907 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02 2023-05-02 02:55:24,908 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02 2023-05-02 02:55:24,910 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-02 02:55:24,911 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 78fb849dbb85ed1018811c0b9ad29f02 2023-05-02 02:55:24,912 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 78fb849dbb85ed1018811c0b9ad29f02; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72096222, jitterRate=0.07431742548942566}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-02 02:55:24,912 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 78fb849dbb85ed1018811c0b9ad29f02: 2023-05-02 02:55:24,913 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,,1682996120349.78fb849dbb85ed1018811c0b9ad29f02., pid=117, masterSystemTime=1682996124830 2023-05-02 02:55:24,913 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-02 02:55:24,915 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-02 02:55:24,915 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-02 02:55:24,916 INFO [RS:2;jenkins-hbase4:42019-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1682996120349.78fb849dbb85ed1018811c0b9ad29f02. 2023-05-02 02:55:24,916 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-02 02:55:24,916 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] regionserver.HStore(1912): 78fb849dbb85ed1018811c0b9ad29f02/cf1 is initiating minor compaction (all files) 2023-05-02 02:55:24,916 INFO [RS:2;jenkins-hbase4:42019-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1682996120349.78fb849dbb85ed1018811c0b9ad29f02. 2023-05-02 02:55:24,916 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] regionserver.HStore(1912): 78fb849dbb85ed1018811c0b9ad29f02/cf2 is initiating minor compaction (all files) 2023-05-02 02:55:24,916 INFO [RS:2;jenkins-hbase4:42019-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 78fb849dbb85ed1018811c0b9ad29f02/cf1 in testRecoveryAndDoubleExecution,,1682996120349.78fb849dbb85ed1018811c0b9ad29f02. 2023-05-02 02:55:24,916 INFO [RS:2;jenkins-hbase4:42019-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 78fb849dbb85ed1018811c0b9ad29f02/cf2 in testRecoveryAndDoubleExecution,,1682996120349.78fb849dbb85ed1018811c0b9ad29f02. 2023-05-02 02:55:24,917 INFO [RS:2;jenkins-hbase4:42019-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf2/78391fd40923430ba069adcd774c9f4b.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/78391fd40923430ba069adcd774c9f4b-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf2/9a1be768ec4e4cdbadc55ffdbfe82828.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/9a1be768ec4e4cdbadc55ffdbfe82828-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf2/b9608c129d224c4fa22a6083bbb53907.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/b9608c129d224c4fa22a6083bbb53907-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf2/902bebca210f45908626f299bace6411.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/902bebca210f45908626f299bace6411-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf2/4ddd3ec587eb4d9f9c2d89d53cdc963f.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/4ddd3ec587eb4d9f9c2d89d53cdc963f-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf2/b3d0ee38ca69423f9a6877578518bc76.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/b3d0ee38ca69423f9a6877578518bc76-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf2/b8675e8553444d2bbde903dca91aba1e.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/b8675e8553444d2bbde903dca91aba1e-bottom] into tmpdir=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/.tmp, totalSize=34.9 K 2023-05-02 02:55:24,917 INFO [RS:2;jenkins-hbase4:42019-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf1/84b2009af5f3401287c117850ef9fc4b.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/84b2009af5f3401287c117850ef9fc4b-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf1/bc695fd948ff4e53b03206a746904fa3.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/bc695fd948ff4e53b03206a746904fa3-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf1/e5774ef6b56443f583eeef35a0086dd7.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/e5774ef6b56443f583eeef35a0086dd7-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf1/8f67930f70934c80893ea92e3da97da1.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/8f67930f70934c80893ea92e3da97da1-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf1/0837374a7f2a4c82bfa97443781ee242.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/0837374a7f2a4c82bfa97443781ee242-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf1/b1149ff9cf704fc0b06d4a93d73229dc.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/b1149ff9cf704fc0b06d4a93d73229dc-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf1/21a49995122e48a493b6d0a8abb3ff52.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/21a49995122e48a493b6d0a8abb3ff52-bottom] into tmpdir=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/.tmp, totalSize=34.9 K 2023-05-02 02:55:24,917 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] compactions.Compactor(207): Compacting 78391fd40923430ba069adcd774c9f4b.059f75e2e092770bf7a63b32739cb003, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1682996118994 2023-05-02 02:55:24,917 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] compactions.Compactor(207): Compacting 84b2009af5f3401287c117850ef9fc4b.059f75e2e092770bf7a63b32739cb003, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1682996118994 2023-05-02 02:55:24,918 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] compactions.Compactor(207): Compacting 9a1be768ec4e4cdbadc55ffdbfe82828.059f75e2e092770bf7a63b32739cb003, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1682996119209 2023-05-02 02:55:24,918 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] compactions.Compactor(207): Compacting bc695fd948ff4e53b03206a746904fa3.059f75e2e092770bf7a63b32739cb003, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1682996119209 2023-05-02 02:55:24,918 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] compactions.Compactor(207): Compacting b9608c129d224c4fa22a6083bbb53907.059f75e2e092770bf7a63b32739cb003, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1682996119380 2023-05-02 02:55:24,918 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] compactions.Compactor(207): Compacting e5774ef6b56443f583eeef35a0086dd7.059f75e2e092770bf7a63b32739cb003, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1682996119380 2023-05-02 02:55:24,919 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,,1682996120349.78fb849dbb85ed1018811c0b9ad29f02. 2023-05-02 02:55:24,919 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRecoveryAndDoubleExecution,,1682996120349.78fb849dbb85ed1018811c0b9ad29f02. 2023-05-02 02:55:24,919 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRecoveryAndDoubleExecution,41,1682996120349.f920d56a4f8ae9570632b12801c0f779. 2023-05-02 02:55:24,919 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => f920d56a4f8ae9570632b12801c0f779, NAME => 'testRecoveryAndDoubleExecution,41,1682996120349.f920d56a4f8ae9570632b12801c0f779.', STARTKEY => '41', ENDKEY => ''} 2023-05-02 02:55:24,919 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=78fb849dbb85ed1018811c0b9ad29f02, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:24,919 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1682996120349.78fb849dbb85ed1018811c0b9ad29f02.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1682996124919"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1682996124919"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1682996124919"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682996124919"}]},"ts":"1682996124919"} 2023-05-02 02:55:24,919 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-02 02:55:24,919 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:24,920 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] compactions.Compactor(207): Compacting 902bebca210f45908626f299bace6411.059f75e2e092770bf7a63b32739cb003, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1682996119562 2023-05-02 02:55:24,920 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] compactions.Compactor(207): Compacting 8f67930f70934c80893ea92e3da97da1.059f75e2e092770bf7a63b32739cb003, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1682996119562 2023-05-02 02:55:24,920 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRecoveryAndDoubleExecution,41,1682996120349.f920d56a4f8ae9570632b12801c0f779.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-02 02:55:24,920 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:24,920 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:24,920 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] compactions.Compactor(207): Compacting 4ddd3ec587eb4d9f9c2d89d53cdc963f.059f75e2e092770bf7a63b32739cb003, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1682996119732 2023-05-02 02:55:24,920 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] compactions.Compactor(207): Compacting 0837374a7f2a4c82bfa97443781ee242.059f75e2e092770bf7a63b32739cb003, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1682996119732 2023-05-02 02:55:24,921 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] compactions.Compactor(207): Compacting b1149ff9cf704fc0b06d4a93d73229dc.059f75e2e092770bf7a63b32739cb003, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1682996119916 2023-05-02 02:55:24,922 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] compactions.Compactor(207): Compacting b3d0ee38ca69423f9a6877578518bc76.059f75e2e092770bf7a63b32739cb003, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1682996119916 2023-05-02 02:55:24,922 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] compactions.Compactor(207): Compacting 21a49995122e48a493b6d0a8abb3ff52.059f75e2e092770bf7a63b32739cb003, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1682996120090 2023-05-02 02:55:24,922 INFO [StoreOpener-f920d56a4f8ae9570632b12801c0f779-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:24,922 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=117, resume processing ppid=115 2023-05-02 02:55:24,922 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=117, ppid=115, state=SUCCESS; OpenRegionProcedure 78fb849dbb85ed1018811c0b9ad29f02, server=jenkins-hbase4.apache.org,42019,1682996021676 in 241 msec 2023-05-02 02:55:24,923 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] compactions.Compactor(207): Compacting b8675e8553444d2bbde903dca91aba1e.059f75e2e092770bf7a63b32739cb003, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1682996120090 2023-05-02 02:55:24,923 DEBUG [StoreOpener-f920d56a4f8ae9570632b12801c0f779-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf1 2023-05-02 02:55:24,924 DEBUG [StoreOpener-f920d56a4f8ae9570632b12801c0f779-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf1 2023-05-02 02:55:24,924 INFO [StoreOpener-f920d56a4f8ae9570632b12801c0f779-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f920d56a4f8ae9570632b12801c0f779 columnFamilyName cf1 2023-05-02 02:55:24,924 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=78fb849dbb85ed1018811c0b9ad29f02, ASSIGN in 400 msec 2023-05-02 02:55:24,936 INFO [RS:2;jenkins-hbase4:42019-shortCompactions-0] throttle.PressureAwareThroughputController(145): 78fb849dbb85ed1018811c0b9ad29f02#cf1#compaction#108 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-02 02:55:24,940 INFO [RS:2;jenkins-hbase4:42019-longCompactions-0] throttle.PressureAwareThroughputController(145): 78fb849dbb85ed1018811c0b9ad29f02#cf2#compaction#109 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-02 02:55:24,947 DEBUG [StoreOpener-f920d56a4f8ae9570632b12801c0f779-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf1/0837374a7f2a4c82bfa97443781ee242.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/0837374a7f2a4c82bfa97443781ee242-top 2023-05-02 02:55:24,950 DEBUG [StoreOpener-f920d56a4f8ae9570632b12801c0f779-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf1/21a49995122e48a493b6d0a8abb3ff52.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/21a49995122e48a493b6d0a8abb3ff52-top 2023-05-02 02:55:24,958 DEBUG [StoreOpener-f920d56a4f8ae9570632b12801c0f779-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf1/84b2009af5f3401287c117850ef9fc4b.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/84b2009af5f3401287c117850ef9fc4b-top 2023-05-02 02:55:24,962 DEBUG [StoreOpener-f920d56a4f8ae9570632b12801c0f779-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf1/8f67930f70934c80893ea92e3da97da1.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/8f67930f70934c80893ea92e3da97da1-top 2023-05-02 02:55:24,966 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/.tmp/cf1/955625a134714786a0f4faf66d4cb5ca as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf1/955625a134714786a0f4faf66d4cb5ca 2023-05-02 02:55:24,966 DEBUG [StoreOpener-f920d56a4f8ae9570632b12801c0f779-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf1/b1149ff9cf704fc0b06d4a93d73229dc.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/b1149ff9cf704fc0b06d4a93d73229dc-top 2023-05-02 02:55:24,968 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/.tmp/cf2/f53ee6b32b954757a07448983421d641 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf2/f53ee6b32b954757a07448983421d641 2023-05-02 02:55:24,972 INFO [RS:2;jenkins-hbase4:42019-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 78fb849dbb85ed1018811c0b9ad29f02/cf1 of 78fb849dbb85ed1018811c0b9ad29f02 into 955625a134714786a0f4faf66d4cb5ca(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-02 02:55:24,972 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 78fb849dbb85ed1018811c0b9ad29f02: 2023-05-02 02:55:24,972 INFO [RS:2;jenkins-hbase4:42019-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1682996120349.78fb849dbb85ed1018811c0b9ad29f02., storeName=78fb849dbb85ed1018811c0b9ad29f02/cf1, priority=9, startTime=1682996124913; duration=0sec 2023-05-02 02:55:24,973 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-02 02:55:24,974 DEBUG [StoreOpener-f920d56a4f8ae9570632b12801c0f779-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf1/bc695fd948ff4e53b03206a746904fa3.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/bc695fd948ff4e53b03206a746904fa3-top 2023-05-02 02:55:24,975 INFO [RS:2;jenkins-hbase4:42019-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 78fb849dbb85ed1018811c0b9ad29f02/cf2 of 78fb849dbb85ed1018811c0b9ad29f02 into f53ee6b32b954757a07448983421d641(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-02 02:55:24,975 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 78fb849dbb85ed1018811c0b9ad29f02: 2023-05-02 02:55:24,975 INFO [RS:2;jenkins-hbase4:42019-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1682996120349.78fb849dbb85ed1018811c0b9ad29f02., storeName=78fb849dbb85ed1018811c0b9ad29f02/cf2, priority=9, startTime=1682996124915; duration=0sec 2023-05-02 02:55:24,975 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-02 02:55:24,977 DEBUG [StoreOpener-f920d56a4f8ae9570632b12801c0f779-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf1/e5774ef6b56443f583eeef35a0086dd7.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/e5774ef6b56443f583eeef35a0086dd7-top 2023-05-02 02:55:24,977 INFO [StoreOpener-f920d56a4f8ae9570632b12801c0f779-1] regionserver.HStore(310): Store=f920d56a4f8ae9570632b12801c0f779/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-02 02:55:24,977 INFO [StoreOpener-f920d56a4f8ae9570632b12801c0f779-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:24,978 DEBUG [StoreOpener-f920d56a4f8ae9570632b12801c0f779-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf2 2023-05-02 02:55:24,978 DEBUG [StoreOpener-f920d56a4f8ae9570632b12801c0f779-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf2 2023-05-02 02:55:24,978 INFO [StoreOpener-f920d56a4f8ae9570632b12801c0f779-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f920d56a4f8ae9570632b12801c0f779 columnFamilyName cf2 2023-05-02 02:55:24,989 DEBUG [StoreOpener-f920d56a4f8ae9570632b12801c0f779-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf2/4ddd3ec587eb4d9f9c2d89d53cdc963f.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/4ddd3ec587eb4d9f9c2d89d53cdc963f-top 2023-05-02 02:55:24,993 DEBUG [StoreOpener-f920d56a4f8ae9570632b12801c0f779-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf2/78391fd40923430ba069adcd774c9f4b.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/78391fd40923430ba069adcd774c9f4b-top 2023-05-02 02:55:24,996 DEBUG [StoreOpener-f920d56a4f8ae9570632b12801c0f779-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf2/902bebca210f45908626f299bace6411.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/902bebca210f45908626f299bace6411-top 2023-05-02 02:55:24,999 DEBUG [StoreOpener-f920d56a4f8ae9570632b12801c0f779-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf2/9a1be768ec4e4cdbadc55ffdbfe82828.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/9a1be768ec4e4cdbadc55ffdbfe82828-top 2023-05-02 02:55:25,003 DEBUG [StoreOpener-f920d56a4f8ae9570632b12801c0f779-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf2/b3d0ee38ca69423f9a6877578518bc76.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/b3d0ee38ca69423f9a6877578518bc76-top 2023-05-02 02:55:25,005 DEBUG [StoreOpener-f920d56a4f8ae9570632b12801c0f779-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf2/b8675e8553444d2bbde903dca91aba1e.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/b8675e8553444d2bbde903dca91aba1e-top 2023-05-02 02:55:25,008 DEBUG [StoreOpener-f920d56a4f8ae9570632b12801c0f779-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf2/b9608c129d224c4fa22a6083bbb53907.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/b9608c129d224c4fa22a6083bbb53907-top 2023-05-02 02:55:25,009 INFO [StoreOpener-f920d56a4f8ae9570632b12801c0f779-1] regionserver.HStore(310): Store=f920d56a4f8ae9570632b12801c0f779/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-02 02:55:25,009 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:25,010 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:25,011 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-02 02:55:25,012 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:25,013 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened f920d56a4f8ae9570632b12801c0f779; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60165523, jitterRate=-0.10346384346485138}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-02 02:55:25,013 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for f920d56a4f8ae9570632b12801c0f779: 2023-05-02 02:55:25,014 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,41,1682996120349.f920d56a4f8ae9570632b12801c0f779., pid=118, masterSystemTime=1682996124830 2023-05-02 02:55:25,014 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-02 02:55:25,015 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-05-02 02:55:25,015 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-02 02:55:25,015 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-02 02:55:25,015 INFO [RS:2;jenkins-hbase4:42019-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1682996120349.f920d56a4f8ae9570632b12801c0f779. 2023-05-02 02:55:25,015 INFO [RS:2;jenkins-hbase4:42019-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1682996120349.f920d56a4f8ae9570632b12801c0f779. 2023-05-02 02:55:25,015 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] regionserver.HStore(1912): f920d56a4f8ae9570632b12801c0f779/cf1 is initiating minor compaction (all files) 2023-05-02 02:55:25,015 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] regionserver.HStore(1912): f920d56a4f8ae9570632b12801c0f779/cf2 is initiating minor compaction (all files) 2023-05-02 02:55:25,015 INFO [RS:2;jenkins-hbase4:42019-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of f920d56a4f8ae9570632b12801c0f779/cf1 in testRecoveryAndDoubleExecution,41,1682996120349.f920d56a4f8ae9570632b12801c0f779. 2023-05-02 02:55:25,015 INFO [RS:2;jenkins-hbase4:42019-longCompactions-0] regionserver.HRegion(2259): Starting compaction of f920d56a4f8ae9570632b12801c0f779/cf2 in testRecoveryAndDoubleExecution,41,1682996120349.f920d56a4f8ae9570632b12801c0f779. 2023-05-02 02:55:25,015 INFO [RS:2;jenkins-hbase4:42019-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf1/84b2009af5f3401287c117850ef9fc4b.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/84b2009af5f3401287c117850ef9fc4b-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf1/bc695fd948ff4e53b03206a746904fa3.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/bc695fd948ff4e53b03206a746904fa3-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf1/e5774ef6b56443f583eeef35a0086dd7.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/e5774ef6b56443f583eeef35a0086dd7-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf1/8f67930f70934c80893ea92e3da97da1.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/8f67930f70934c80893ea92e3da97da1-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf1/0837374a7f2a4c82bfa97443781ee242.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/0837374a7f2a4c82bfa97443781ee242-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf1/b1149ff9cf704fc0b06d4a93d73229dc.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/b1149ff9cf704fc0b06d4a93d73229dc-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf1/21a49995122e48a493b6d0a8abb3ff52.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/21a49995122e48a493b6d0a8abb3ff52-top] into tmpdir=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/.tmp, totalSize=34.9 K 2023-05-02 02:55:25,016 INFO [RS:2;jenkins-hbase4:42019-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf2/78391fd40923430ba069adcd774c9f4b.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/78391fd40923430ba069adcd774c9f4b-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf2/9a1be768ec4e4cdbadc55ffdbfe82828.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/9a1be768ec4e4cdbadc55ffdbfe82828-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf2/b9608c129d224c4fa22a6083bbb53907.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/b9608c129d224c4fa22a6083bbb53907-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf2/902bebca210f45908626f299bace6411.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/902bebca210f45908626f299bace6411-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf2/4ddd3ec587eb4d9f9c2d89d53cdc963f.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/4ddd3ec587eb4d9f9c2d89d53cdc963f-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf2/b3d0ee38ca69423f9a6877578518bc76.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/b3d0ee38ca69423f9a6877578518bc76-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf2/b8675e8553444d2bbde903dca91aba1e.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/b8675e8553444d2bbde903dca91aba1e-top] into tmpdir=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/.tmp, totalSize=34.9 K 2023-05-02 02:55:25,016 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] compactions.Compactor(207): Compacting 84b2009af5f3401287c117850ef9fc4b.059f75e2e092770bf7a63b32739cb003, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1682996118994 2023-05-02 02:55:25,016 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] compactions.Compactor(207): Compacting 78391fd40923430ba069adcd774c9f4b.059f75e2e092770bf7a63b32739cb003, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1682996118994 2023-05-02 02:55:25,016 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,41,1682996120349.f920d56a4f8ae9570632b12801c0f779. 2023-05-02 02:55:25,016 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRecoveryAndDoubleExecution,41,1682996120349.f920d56a4f8ae9570632b12801c0f779. 2023-05-02 02:55:25,017 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] compactions.Compactor(207): Compacting bc695fd948ff4e53b03206a746904fa3.059f75e2e092770bf7a63b32739cb003, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1682996119209 2023-05-02 02:55:25,017 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=f920d56a4f8ae9570632b12801c0f779, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:25,017 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] compactions.Compactor(207): Compacting 9a1be768ec4e4cdbadc55ffdbfe82828.059f75e2e092770bf7a63b32739cb003, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1682996119209 2023-05-02 02:55:25,017 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,41,1682996120349.f920d56a4f8ae9570632b12801c0f779.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1682996125017"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1682996125017"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1682996125017"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682996125017"}]},"ts":"1682996125017"} 2023-05-02 02:55:25,017 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] compactions.Compactor(207): Compacting e5774ef6b56443f583eeef35a0086dd7.059f75e2e092770bf7a63b32739cb003, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1682996119380 2023-05-02 02:55:25,017 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] compactions.Compactor(207): Compacting b9608c129d224c4fa22a6083bbb53907.059f75e2e092770bf7a63b32739cb003, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1682996119380 2023-05-02 02:55:25,017 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] compactions.Compactor(207): Compacting 8f67930f70934c80893ea92e3da97da1.059f75e2e092770bf7a63b32739cb003, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1682996119562 2023-05-02 02:55:25,017 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] compactions.Compactor(207): Compacting 902bebca210f45908626f299bace6411.059f75e2e092770bf7a63b32739cb003, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1682996119562 2023-05-02 02:55:25,018 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] compactions.Compactor(207): Compacting 0837374a7f2a4c82bfa97443781ee242.059f75e2e092770bf7a63b32739cb003, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1682996119732 2023-05-02 02:55:25,018 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] compactions.Compactor(207): Compacting 4ddd3ec587eb4d9f9c2d89d53cdc963f.059f75e2e092770bf7a63b32739cb003, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1682996119732 2023-05-02 02:55:25,018 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] compactions.Compactor(207): Compacting b1149ff9cf704fc0b06d4a93d73229dc.059f75e2e092770bf7a63b32739cb003, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1682996119916 2023-05-02 02:55:25,018 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] compactions.Compactor(207): Compacting b3d0ee38ca69423f9a6877578518bc76.059f75e2e092770bf7a63b32739cb003, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1682996119916 2023-05-02 02:55:25,019 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] compactions.Compactor(207): Compacting 21a49995122e48a493b6d0a8abb3ff52.059f75e2e092770bf7a63b32739cb003, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1682996120090 2023-05-02 02:55:25,019 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] compactions.Compactor(207): Compacting b8675e8553444d2bbde903dca91aba1e.059f75e2e092770bf7a63b32739cb003, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1682996120090 2023-05-02 02:55:25,020 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=118, resume processing ppid=116 2023-05-02 02:55:25,020 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=118, ppid=116, state=SUCCESS; OpenRegionProcedure f920d56a4f8ae9570632b12801c0f779, server=jenkins-hbase4.apache.org,42019,1682996021676 in 336 msec 2023-05-02 02:55:25,022 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=116, resume processing ppid=112 2023-05-02 02:55:25,022 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=f920d56a4f8ae9570632b12801c0f779, ASSIGN in 497 msec 2023-05-02 02:55:25,022 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-02 02:55:25,022 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:25,022 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-02 02:55:25,022 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-02 02:55:25,023 INFO [Listener at localhost/37119] procedure.MasterProcedureTestingUtility(454): Restart 10 exec state=pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:25,023 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-02 02:55:25,023 INFO [Listener at localhost/37119] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-02 02:55:25,023 INFO [Listener at localhost/37119] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-02 02:55:25,023 ERROR [Listener at localhost/37119] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-02 02:55:25,024 INFO [Listener at localhost/37119] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-02 02:55:25,024 INFO [Listener at localhost/37119] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-02 02:55:25,024 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-02 02:55:25,024 INFO [Listener at localhost/37119] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-02 02:55:25,024 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-02 02:55:25,026 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12, 13] 2023-05-02 02:55:25,026 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-02 02:55:25,026 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-02 02:55:25,026 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=115, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[15, 16, 21] 2023-05-02 02:55:25,026 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=117, ppid=115, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[17, 20] 2023-05-02 02:55:25,026 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=116, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[14, 18, 23] 2023-05-02 02:55:25,026 DEBUG [Listener at localhost/37119] store.ProcedureTree(114): Procedure Procedure(pid=118, ppid=116, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[19, 22] 2023-05-02 02:55:25,027 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-02 02:55:25,027 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-02 02:55:25,027 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-02 02:55:25,027 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-02 02:55:25,028 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:55:25,028 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=084b1607f7ed8615120a04e4729fbafa, daughterA=ea7683cd3436a62eb16418a3c875d563, daughterB=408e891cab9b919f4038b233ba34779d 2023-05-02 02:55:25,028 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:55:25,028 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-02 02:55:25,029 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-02 02:55:25,029 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=b92c1c7e8ba3ec18a0b4e94867da9855, daughterA=c0e0b3bf0872704f385d48808df6a51e, daughterB=6e3e738dce7ec3e6ddbdadab93b4596e 2023-05-02 02:55:25,029 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-02 02:55:25,029 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-02 02:55:25,029 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-02 02:55:25,030 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=42fc9cc7beccd63fc2605b5f0fb93a8a, daughterA=dc88a1a6241e1b8594ba9db18769364a, daughterB=cd6c9dca49228dd6658df10e124fb142 2023-05-02 02:55:25,030 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=42fc9cc7beccd63fc2605b5f0fb93a8a, ASSIGN 2023-05-02 02:55:25,030 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-05-02 02:55:25,030 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-05-02 02:55:25,030 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-02 02:55:25,031 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=afa57eddfa2d26bf35b12dc022351e78, daughterA=b91a201af734ea9cb01587589c96e908, daughterB=292a312492e2395ec0ca83a160894be9 2023-05-02 02:55:25,031 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-02 02:55:25,031 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-02 02:55:25,031 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:55:25,031 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=8af7d4111b21c70efea08f7beab73a9f, daughterA=bb895e1df53cda46bfcb5f94fe1710c9, daughterB=024f19513d3a9ded6c060054f1607e68 2023-05-02 02:55:25,031 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:55:25,031 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-02 02:55:25,032 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-02 02:55:25,032 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-02 02:55:25,032 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,39081,1682996021579, table=testRollbackForSplitTableRegionWithReplica, region=1b91efe506b89bc57ba68287b021be4f is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=74b19512a655e6f00fc418d81fba16c8, daughterA=bfa227908c6548a4187e4deb87e808a6, daughterB=da50a5f8e634bd92f99eebe500206397 2023-05-02 02:55:25,032 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-02 02:55:25,033 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-02 02:55:25,033 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-05-02 02:55:25,033 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-05-02 02:55:25,033 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-05-02 02:55:25,033 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-05-02 02:55:25,035 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:25,035 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=059f75e2e092770bf7a63b32739cb003, UNASSIGN 2023-05-02 02:55:25,035 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 059f75e2e092770bf7a63b32739cb003, server=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:25,035 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=78fb849dbb85ed1018811c0b9ad29f02, ASSIGN 2023-05-02 02:55:25,035 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=f920d56a4f8ae9570632b12801c0f779, ASSIGN 2023-05-02 02:55:25,035 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=117, ppid=115, state=SUCCESS; OpenRegionProcedure 78fb849dbb85ed1018811c0b9ad29f02, server=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:25,035 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(468): Loading pid=118, ppid=116, state=SUCCESS; OpenRegionProcedure f920d56a4f8ae9570632b12801c0f779, server=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:25,035 DEBUG [Listener at localhost/37119] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 held the lock before restarting, call acquireLock to restore it. 2023-05-02 02:55:25,036 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:25,036 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:25,036 INFO [Listener at localhost/37119] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:25,036 DEBUG [Listener at localhost/37119] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=059f75e2e092770bf7a63b32739cb003, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-02 02:55:25,036 DEBUG [Listener at localhost/37119] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 059f75e2e092770bf7a63b32739cb003, server=jenkins-hbase4.apache.org,42019,1682996021676 didn't hold the lock before restarting, skip acquiring lock. 2023-05-02 02:55:25,036 INFO [RS:2;jenkins-hbase4:42019-longCompactions-0] throttle.PressureAwareThroughputController(145): f920d56a4f8ae9570632b12801c0f779#cf2#compaction#110 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-02 02:55:25,036 DEBUG [Listener at localhost/37119] procedure2.Procedure(946): pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=78fb849dbb85ed1018811c0b9ad29f02, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-02 02:55:25,036 DEBUG [Listener at localhost/37119] procedure2.Procedure(946): pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=f920d56a4f8ae9570632b12801c0f779, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-02 02:55:25,036 DEBUG [Listener at localhost/37119] procedure2.Procedure(946): pid=117, ppid=115, state=SUCCESS; OpenRegionProcedure 78fb849dbb85ed1018811c0b9ad29f02, server=jenkins-hbase4.apache.org,42019,1682996021676 didn't hold the lock before restarting, skip acquiring lock. 2023-05-02 02:55:25,037 DEBUG [Listener at localhost/37119] procedure2.Procedure(946): pid=118, ppid=116, state=SUCCESS; OpenRegionProcedure f920d56a4f8ae9570632b12801c0f779, server=jenkins-hbase4.apache.org,42019,1682996021676 didn't hold the lock before restarting, skip acquiring lock. 2023-05-02 02:55:25,037 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 12 msec 2023-05-02 02:55:25,037 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-02 02:55:25,038 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,38925,1682996021633, table=hbase:meta, region=1588230740 2023-05-02 02:55:25,040 INFO [RS:2;jenkins-hbase4:42019-shortCompactions-0] throttle.PressureAwareThroughputController(145): f920d56a4f8ae9570632b12801c0f779#cf1#compaction#111 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-02 02:55:25,044 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682996155044 2023-05-02 02:55:25,044 DEBUG [Listener at localhost/37119] assignment.AssignmentManager(1517): Joining cluster... 2023-05-02 02:55:25,053 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:25,053 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=5c823d6f15ad7b8fe82f9ae4beee6513, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42019,1682996021676, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676, openSeqNum=2 2023-05-02 02:55:25,054 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=059f75e2e092770bf7a63b32739cb003, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,42019,1682996021676, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676, openSeqNum=2 2023-05-02 02:55:25,054 INFO [Listener at localhost/37119] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,42019,1682996021676, table=testRecoveryAndDoubleExecution, region=059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:25,054 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=78fb849dbb85ed1018811c0b9ad29f02, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42019,1682996021676, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676, openSeqNum=86 2023-05-02 02:55:25,054 DEBUG [Listener at localhost/37119] assignment.RegionStateStore(147): Load hbase:meta entry region=f920d56a4f8ae9570632b12801c0f779, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42019,1682996021676, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676, openSeqNum=86 2023-05-02 02:55:25,054 INFO [Listener at localhost/37119] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-02 02:55:25,054 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682996185054 2023-05-02 02:55:25,054 INFO [Listener at localhost/37119] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682996245054 2023-05-02 02:55:25,054 INFO [Listener at localhost/37119] assignment.AssignmentManager(1536): Joined the cluster in 10 msec 2023-05-02 02:55:25,056 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-02 02:55:25,065 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=112, state=SUCCESS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=059f75e2e092770bf7a63b32739cb003, daughterA=78fb849dbb85ed1018811c0b9ad29f02, daughterB=f920d56a4f8ae9570632b12801c0f779 in 4.7070 sec 2023-05-02 02:55:25,076 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/.tmp/cf2/8294fbe0408a481997371dfb8c588187 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf2/8294fbe0408a481997371dfb8c588187 2023-05-02 02:55:25,076 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/.tmp/cf1/fafa0db96778417b87b8436090266a5b as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf1/fafa0db96778417b87b8436090266a5b 2023-05-02 02:55:25,082 INFO [RS:2;jenkins-hbase4:42019-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in f920d56a4f8ae9570632b12801c0f779/cf2 of f920d56a4f8ae9570632b12801c0f779 into 8294fbe0408a481997371dfb8c588187(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-02 02:55:25,082 INFO [RS:2;jenkins-hbase4:42019-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in f920d56a4f8ae9570632b12801c0f779/cf1 of f920d56a4f8ae9570632b12801c0f779 into fafa0db96778417b87b8436090266a5b(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-02 02:55:25,082 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for f920d56a4f8ae9570632b12801c0f779: 2023-05-02 02:55:25,082 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for f920d56a4f8ae9570632b12801c0f779: 2023-05-02 02:55:25,082 INFO [RS:2;jenkins-hbase4:42019-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1682996120349.f920d56a4f8ae9570632b12801c0f779., storeName=f920d56a4f8ae9570632b12801c0f779/cf2, priority=9, startTime=1682996125015; duration=0sec 2023-05-02 02:55:25,082 INFO [RS:2;jenkins-hbase4:42019-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1682996120349.f920d56a4f8ae9570632b12801c0f779., storeName=f920d56a4f8ae9570632b12801c0f779/cf1, priority=9, startTime=1682996125014; duration=0sec 2023-05-02 02:55:25,082 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-02 02:55:25,082 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-02 02:55:25,308 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42019] ipc.CallRunner(144): callId: 856 service: ClientService methodName: Scan size: 154 connection: 172.31.14.131:58782 deadline: 1682996185307, exception=org.apache.hadoop.hbase.NotServingRegionException: testRecoveryAndDoubleExecution,,1682996118490.059f75e2e092770bf7a63b32739cb003. is not online on jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:25,419 DEBUG [Listener at localhost/37119] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-05-02 02:55:25,425 INFO [Listener at localhost/37119] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-05-02 02:55:25,427 DEBUG [Listener at localhost/37119] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-05-02 02:55:25,435 WARN [Listener at localhost/37119] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-02 02:55:25,436 INFO [Listener at localhost/37119] client.HBaseAdmin$15(890): Started disable of testRecoveryAndDoubleExecution 2023-05-02 02:55:25,437 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testRecoveryAndDoubleExecution 2023-05-02 02:55:25,437 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure2.ProcedureExecutor(1029): Stored pid=119, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRecoveryAndDoubleExecution 2023-05-02 02:55:25,439 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-05-02 02:55:25,440 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682996125440"}]},"ts":"1682996125440"} 2023-05-02 02:55:25,441 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLING in hbase:meta 2023-05-02 02:55:25,442 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRecoveryAndDoubleExecution to state=DISABLING 2023-05-02 02:55:25,442 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=78fb849dbb85ed1018811c0b9ad29f02, UNASSIGN}, {pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=f920d56a4f8ae9570632b12801c0f779, UNASSIGN}] 2023-05-02 02:55:25,443 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=f920d56a4f8ae9570632b12801c0f779, UNASSIGN 2023-05-02 02:55:25,443 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=f920d56a4f8ae9570632b12801c0f779, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:25,444 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1682996120349.f920d56a4f8ae9570632b12801c0f779.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1682996125443"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1682996125443"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682996125443"}]},"ts":"1682996125443"} 2023-05-02 02:55:25,445 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=122, ppid=121, state=RUNNABLE; CloseRegionProcedure f920d56a4f8ae9570632b12801c0f779, server=jenkins-hbase4.apache.org,42019,1682996021676}] 2023-05-02 02:55:25,446 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=78fb849dbb85ed1018811c0b9ad29f02, UNASSIGN 2023-05-02 02:55:25,447 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=78fb849dbb85ed1018811c0b9ad29f02, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:25,447 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1682996120349.78fb849dbb85ed1018811c0b9ad29f02.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1682996125447"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1682996125447"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682996125447"}]},"ts":"1682996125447"} 2023-05-02 02:55:25,448 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=123, ppid=120, state=RUNNABLE; CloseRegionProcedure 78fb849dbb85ed1018811c0b9ad29f02, server=jenkins-hbase4.apache.org,42019,1682996021676}] 2023-05-02 02:55:25,540 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-05-02 02:55:25,596 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:25,597 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing f920d56a4f8ae9570632b12801c0f779, disabling compactions & flushes 2023-05-02 02:55:25,597 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRecoveryAndDoubleExecution,41,1682996120349.f920d56a4f8ae9570632b12801c0f779. 2023-05-02 02:55:25,597 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,41,1682996120349.f920d56a4f8ae9570632b12801c0f779. 2023-05-02 02:55:25,597 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRecoveryAndDoubleExecution,41,1682996120349.f920d56a4f8ae9570632b12801c0f779. after waiting 0 ms 2023-05-02 02:55:25,597 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRecoveryAndDoubleExecution,41,1682996120349.f920d56a4f8ae9570632b12801c0f779. 2023-05-02 02:55:25,597 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1682996120349.f920d56a4f8ae9570632b12801c0f779.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf1/84b2009af5f3401287c117850ef9fc4b.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/84b2009af5f3401287c117850ef9fc4b-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf1/bc695fd948ff4e53b03206a746904fa3.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/bc695fd948ff4e53b03206a746904fa3-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf1/e5774ef6b56443f583eeef35a0086dd7.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/e5774ef6b56443f583eeef35a0086dd7-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf1/8f67930f70934c80893ea92e3da97da1.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/8f67930f70934c80893ea92e3da97da1-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf1/0837374a7f2a4c82bfa97443781ee242.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/0837374a7f2a4c82bfa97443781ee242-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf1/b1149ff9cf704fc0b06d4a93d73229dc.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/b1149ff9cf704fc0b06d4a93d73229dc-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf1/21a49995122e48a493b6d0a8abb3ff52.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/21a49995122e48a493b6d0a8abb3ff52-top] to archive 2023-05-02 02:55:25,598 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1682996120349.f920d56a4f8ae9570632b12801c0f779.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-02 02:55:25,600 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1682996120349.f920d56a4f8ae9570632b12801c0f779.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf1/84b2009af5f3401287c117850ef9fc4b.059f75e2e092770bf7a63b32739cb003 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf1/84b2009af5f3401287c117850ef9fc4b.059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:25,601 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1682996120349.f920d56a4f8ae9570632b12801c0f779.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf1/bc695fd948ff4e53b03206a746904fa3.059f75e2e092770bf7a63b32739cb003 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf1/bc695fd948ff4e53b03206a746904fa3.059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:25,602 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1682996120349.f920d56a4f8ae9570632b12801c0f779.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf1/e5774ef6b56443f583eeef35a0086dd7.059f75e2e092770bf7a63b32739cb003 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf1/e5774ef6b56443f583eeef35a0086dd7.059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:25,603 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1682996120349.f920d56a4f8ae9570632b12801c0f779.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf1/8f67930f70934c80893ea92e3da97da1.059f75e2e092770bf7a63b32739cb003 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf1/8f67930f70934c80893ea92e3da97da1.059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:25,604 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1682996120349.f920d56a4f8ae9570632b12801c0f779.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf1/0837374a7f2a4c82bfa97443781ee242.059f75e2e092770bf7a63b32739cb003 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf1/0837374a7f2a4c82bfa97443781ee242.059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:25,605 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1682996120349.f920d56a4f8ae9570632b12801c0f779.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf1/b1149ff9cf704fc0b06d4a93d73229dc.059f75e2e092770bf7a63b32739cb003 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf1/b1149ff9cf704fc0b06d4a93d73229dc.059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:25,606 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1682996120349.f920d56a4f8ae9570632b12801c0f779.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf1/21a49995122e48a493b6d0a8abb3ff52.059f75e2e092770bf7a63b32739cb003 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf1/21a49995122e48a493b6d0a8abb3ff52.059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:25,607 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1682996120349.f920d56a4f8ae9570632b12801c0f779.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf2/78391fd40923430ba069adcd774c9f4b.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/78391fd40923430ba069adcd774c9f4b-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf2/9a1be768ec4e4cdbadc55ffdbfe82828.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/9a1be768ec4e4cdbadc55ffdbfe82828-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf2/b9608c129d224c4fa22a6083bbb53907.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/b9608c129d224c4fa22a6083bbb53907-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf2/902bebca210f45908626f299bace6411.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/902bebca210f45908626f299bace6411-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf2/4ddd3ec587eb4d9f9c2d89d53cdc963f.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/4ddd3ec587eb4d9f9c2d89d53cdc963f-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf2/b3d0ee38ca69423f9a6877578518bc76.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/b3d0ee38ca69423f9a6877578518bc76-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf2/b8675e8553444d2bbde903dca91aba1e.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/b8675e8553444d2bbde903dca91aba1e-top] to archive 2023-05-02 02:55:25,607 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1682996120349.f920d56a4f8ae9570632b12801c0f779.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-02 02:55:25,608 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1682996120349.f920d56a4f8ae9570632b12801c0f779.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf2/78391fd40923430ba069adcd774c9f4b.059f75e2e092770bf7a63b32739cb003 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf2/78391fd40923430ba069adcd774c9f4b.059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:25,609 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1682996120349.f920d56a4f8ae9570632b12801c0f779.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf2/9a1be768ec4e4cdbadc55ffdbfe82828.059f75e2e092770bf7a63b32739cb003 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf2/9a1be768ec4e4cdbadc55ffdbfe82828.059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:25,610 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1682996120349.f920d56a4f8ae9570632b12801c0f779.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf2/b9608c129d224c4fa22a6083bbb53907.059f75e2e092770bf7a63b32739cb003 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf2/b9608c129d224c4fa22a6083bbb53907.059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:25,611 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1682996120349.f920d56a4f8ae9570632b12801c0f779.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf2/902bebca210f45908626f299bace6411.059f75e2e092770bf7a63b32739cb003 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf2/902bebca210f45908626f299bace6411.059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:25,612 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1682996120349.f920d56a4f8ae9570632b12801c0f779.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf2/4ddd3ec587eb4d9f9c2d89d53cdc963f.059f75e2e092770bf7a63b32739cb003 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf2/4ddd3ec587eb4d9f9c2d89d53cdc963f.059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:25,613 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1682996120349.f920d56a4f8ae9570632b12801c0f779.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf2/b3d0ee38ca69423f9a6877578518bc76.059f75e2e092770bf7a63b32739cb003 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf2/b3d0ee38ca69423f9a6877578518bc76.059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:25,613 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1682996120349.f920d56a4f8ae9570632b12801c0f779.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf2/b8675e8553444d2bbde903dca91aba1e.059f75e2e092770bf7a63b32739cb003 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf2/b8675e8553444d2bbde903dca91aba1e.059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:25,617 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-05-02 02:55:25,617 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-02 02:55:25,617 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRecoveryAndDoubleExecution,41,1682996120349.f920d56a4f8ae9570632b12801c0f779. 2023-05-02 02:55:25,617 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for f920d56a4f8ae9570632b12801c0f779: 2023-05-02 02:55:25,619 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:25,619 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 78fb849dbb85ed1018811c0b9ad29f02 2023-05-02 02:55:25,619 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 78fb849dbb85ed1018811c0b9ad29f02, disabling compactions & flushes 2023-05-02 02:55:25,619 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRecoveryAndDoubleExecution,,1682996120349.78fb849dbb85ed1018811c0b9ad29f02. 2023-05-02 02:55:25,619 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1682996120349.78fb849dbb85ed1018811c0b9ad29f02. 2023-05-02 02:55:25,620 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRecoveryAndDoubleExecution,,1682996120349.78fb849dbb85ed1018811c0b9ad29f02. after waiting 0 ms 2023-05-02 02:55:25,620 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRecoveryAndDoubleExecution,,1682996120349.78fb849dbb85ed1018811c0b9ad29f02. 2023-05-02 02:55:25,620 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=f920d56a4f8ae9570632b12801c0f779, regionState=CLOSED 2023-05-02 02:55:25,620 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,41,1682996120349.f920d56a4f8ae9570632b12801c0f779.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1682996125620"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682996125620"}]},"ts":"1682996125620"} 2023-05-02 02:55:25,621 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1682996120349.78fb849dbb85ed1018811c0b9ad29f02.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf1/84b2009af5f3401287c117850ef9fc4b.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/84b2009af5f3401287c117850ef9fc4b-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf1/bc695fd948ff4e53b03206a746904fa3.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/bc695fd948ff4e53b03206a746904fa3-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf1/e5774ef6b56443f583eeef35a0086dd7.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/e5774ef6b56443f583eeef35a0086dd7-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf1/8f67930f70934c80893ea92e3da97da1.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/8f67930f70934c80893ea92e3da97da1-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf1/0837374a7f2a4c82bfa97443781ee242.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/0837374a7f2a4c82bfa97443781ee242-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf1/b1149ff9cf704fc0b06d4a93d73229dc.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/b1149ff9cf704fc0b06d4a93d73229dc-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf1/21a49995122e48a493b6d0a8abb3ff52.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/21a49995122e48a493b6d0a8abb3ff52-bottom] to archive 2023-05-02 02:55:25,621 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1682996120349.78fb849dbb85ed1018811c0b9ad29f02.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-02 02:55:25,623 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=122, resume processing ppid=121 2023-05-02 02:55:25,623 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=122, ppid=121, state=SUCCESS; CloseRegionProcedure f920d56a4f8ae9570632b12801c0f779, server=jenkins-hbase4.apache.org,42019,1682996021676 in 176 msec 2023-05-02 02:55:25,623 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1682996120349.78fb849dbb85ed1018811c0b9ad29f02.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf1/84b2009af5f3401287c117850ef9fc4b.059f75e2e092770bf7a63b32739cb003 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf1/84b2009af5f3401287c117850ef9fc4b.059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:25,624 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=121, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=f920d56a4f8ae9570632b12801c0f779, UNASSIGN in 181 msec 2023-05-02 02:55:25,624 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1682996120349.78fb849dbb85ed1018811c0b9ad29f02.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf1/bc695fd948ff4e53b03206a746904fa3.059f75e2e092770bf7a63b32739cb003 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf1/bc695fd948ff4e53b03206a746904fa3.059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:25,625 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1682996120349.78fb849dbb85ed1018811c0b9ad29f02.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf1/e5774ef6b56443f583eeef35a0086dd7.059f75e2e092770bf7a63b32739cb003 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf1/e5774ef6b56443f583eeef35a0086dd7.059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:25,626 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1682996120349.78fb849dbb85ed1018811c0b9ad29f02.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf1/8f67930f70934c80893ea92e3da97da1.059f75e2e092770bf7a63b32739cb003 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf1/8f67930f70934c80893ea92e3da97da1.059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:25,627 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1682996120349.78fb849dbb85ed1018811c0b9ad29f02.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf1/0837374a7f2a4c82bfa97443781ee242.059f75e2e092770bf7a63b32739cb003 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf1/0837374a7f2a4c82bfa97443781ee242.059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:25,628 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1682996120349.78fb849dbb85ed1018811c0b9ad29f02.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf1/b1149ff9cf704fc0b06d4a93d73229dc.059f75e2e092770bf7a63b32739cb003 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf1/b1149ff9cf704fc0b06d4a93d73229dc.059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:25,629 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1682996120349.78fb849dbb85ed1018811c0b9ad29f02.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf1/21a49995122e48a493b6d0a8abb3ff52.059f75e2e092770bf7a63b32739cb003 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf1/21a49995122e48a493b6d0a8abb3ff52.059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:25,631 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1682996120349.78fb849dbb85ed1018811c0b9ad29f02.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf2/78391fd40923430ba069adcd774c9f4b.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/78391fd40923430ba069adcd774c9f4b-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf2/9a1be768ec4e4cdbadc55ffdbfe82828.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/9a1be768ec4e4cdbadc55ffdbfe82828-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf2/b9608c129d224c4fa22a6083bbb53907.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/b9608c129d224c4fa22a6083bbb53907-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf2/902bebca210f45908626f299bace6411.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/902bebca210f45908626f299bace6411-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf2/4ddd3ec587eb4d9f9c2d89d53cdc963f.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/4ddd3ec587eb4d9f9c2d89d53cdc963f-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf2/b3d0ee38ca69423f9a6877578518bc76.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/b3d0ee38ca69423f9a6877578518bc76-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf2/b8675e8553444d2bbde903dca91aba1e.059f75e2e092770bf7a63b32739cb003->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/b8675e8553444d2bbde903dca91aba1e-bottom] to archive 2023-05-02 02:55:25,631 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1682996120349.78fb849dbb85ed1018811c0b9ad29f02.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-02 02:55:25,632 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1682996120349.78fb849dbb85ed1018811c0b9ad29f02.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf2/78391fd40923430ba069adcd774c9f4b.059f75e2e092770bf7a63b32739cb003 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf2/78391fd40923430ba069adcd774c9f4b.059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:25,633 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1682996120349.78fb849dbb85ed1018811c0b9ad29f02.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf2/9a1be768ec4e4cdbadc55ffdbfe82828.059f75e2e092770bf7a63b32739cb003 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf2/9a1be768ec4e4cdbadc55ffdbfe82828.059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:25,634 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1682996120349.78fb849dbb85ed1018811c0b9ad29f02.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf2/b9608c129d224c4fa22a6083bbb53907.059f75e2e092770bf7a63b32739cb003 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf2/b9608c129d224c4fa22a6083bbb53907.059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:25,635 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1682996120349.78fb849dbb85ed1018811c0b9ad29f02.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf2/902bebca210f45908626f299bace6411.059f75e2e092770bf7a63b32739cb003 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf2/902bebca210f45908626f299bace6411.059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:25,636 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1682996120349.78fb849dbb85ed1018811c0b9ad29f02.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf2/4ddd3ec587eb4d9f9c2d89d53cdc963f.059f75e2e092770bf7a63b32739cb003 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf2/4ddd3ec587eb4d9f9c2d89d53cdc963f.059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:25,637 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1682996120349.78fb849dbb85ed1018811c0b9ad29f02.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf2/b3d0ee38ca69423f9a6877578518bc76.059f75e2e092770bf7a63b32739cb003 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf2/b3d0ee38ca69423f9a6877578518bc76.059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:25,638 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1682996120349.78fb849dbb85ed1018811c0b9ad29f02.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf2/b8675e8553444d2bbde903dca91aba1e.059f75e2e092770bf7a63b32739cb003 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf2/b8675e8553444d2bbde903dca91aba1e.059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:25,641 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-05-02 02:55:25,641 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-02 02:55:25,641 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRecoveryAndDoubleExecution,,1682996120349.78fb849dbb85ed1018811c0b9ad29f02. 2023-05-02 02:55:25,641 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 78fb849dbb85ed1018811c0b9ad29f02: 2023-05-02 02:55:25,643 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 78fb849dbb85ed1018811c0b9ad29f02 2023-05-02 02:55:25,643 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=78fb849dbb85ed1018811c0b9ad29f02, regionState=CLOSED 2023-05-02 02:55:25,644 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1682996120349.78fb849dbb85ed1018811c0b9ad29f02.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1682996125643"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682996125643"}]},"ts":"1682996125643"} 2023-05-02 02:55:25,646 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=123, resume processing ppid=120 2023-05-02 02:55:25,646 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=123, ppid=120, state=SUCCESS; CloseRegionProcedure 78fb849dbb85ed1018811c0b9ad29f02, server=jenkins-hbase4.apache.org,42019,1682996021676 in 197 msec 2023-05-02 02:55:25,648 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=120, resume processing ppid=119 2023-05-02 02:55:25,648 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=120, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=78fb849dbb85ed1018811c0b9ad29f02, UNASSIGN in 204 msec 2023-05-02 02:55:25,650 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682996125650"}]},"ts":"1682996125650"} 2023-05-02 02:55:25,651 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLED in hbase:meta 2023-05-02 02:55:25,652 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRecoveryAndDoubleExecution to state=DISABLED 2023-05-02 02:55:25,654 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=119, state=SUCCESS; DisableTableProcedure table=testRecoveryAndDoubleExecution in 216 msec 2023-05-02 02:55:25,741 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-05-02 02:55:25,742 INFO [Listener at localhost/37119] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRecoveryAndDoubleExecution, procId: 119 completed 2023-05-02 02:55:25,742 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testRecoveryAndDoubleExecution 2023-05-02 02:55:25,743 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure2.ProcedureExecutor(1029): Stored pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-05-02 02:55:25,745 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-05-02 02:55:25,745 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1227): Checking to see if procedure is done pid=124 2023-05-02 02:55:25,745 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=124, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-05-02 02:55:25,749 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:25,749 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02 2023-05-02 02:55:25,750 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:25,752 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1, FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2, FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/recovered.edits] 2023-05-02 02:55:25,752 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf1, FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf2, FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/f920d56a4f8ae9570632b12801c0f779, FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/recovered.edits] 2023-05-02 02:55:25,753 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/78fb849dbb85ed1018811c0b9ad29f02, FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf1, FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf2, FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/recovered.edits] 2023-05-02 02:55:25,756 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf1/fafa0db96778417b87b8436090266a5b to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf1/fafa0db96778417b87b8436090266a5b 2023-05-02 02:55:25,756 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/0837374a7f2a4c82bfa97443781ee242 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/0837374a7f2a4c82bfa97443781ee242 2023-05-02 02:55:25,758 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/21a49995122e48a493b6d0a8abb3ff52 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/21a49995122e48a493b6d0a8abb3ff52 2023-05-02 02:55:25,758 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/78fb849dbb85ed1018811c0b9ad29f02/.regioninfo to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/78fb849dbb85ed1018811c0b9ad29f02/.regioninfo 2023-05-02 02:55:25,759 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf2/8294fbe0408a481997371dfb8c588187 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/cf2/8294fbe0408a481997371dfb8c588187 2023-05-02 02:55:25,759 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/84b2009af5f3401287c117850ef9fc4b to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/84b2009af5f3401287c117850ef9fc4b 2023-05-02 02:55:25,761 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/8f67930f70934c80893ea92e3da97da1 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/8f67930f70934c80893ea92e3da97da1 2023-05-02 02:55:25,762 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/78fb849dbb85ed1018811c0b9ad29f02/cf1/0837374a7f2a4c82bfa97443781ee242.059f75e2e092770bf7a63b32739cb003 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/78fb849dbb85ed1018811c0b9ad29f02/cf1/0837374a7f2a4c82bfa97443781ee242.059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:25,762 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/f920d56a4f8ae9570632b12801c0f779/.regioninfo to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/f920d56a4f8ae9570632b12801c0f779/.regioninfo 2023-05-02 02:55:25,762 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/b1149ff9cf704fc0b06d4a93d73229dc to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/b1149ff9cf704fc0b06d4a93d73229dc 2023-05-02 02:55:25,763 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/78fb849dbb85ed1018811c0b9ad29f02/cf1/21a49995122e48a493b6d0a8abb3ff52.059f75e2e092770bf7a63b32739cb003 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/78fb849dbb85ed1018811c0b9ad29f02/cf1/21a49995122e48a493b6d0a8abb3ff52.059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:25,764 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/bc695fd948ff4e53b03206a746904fa3 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/bc695fd948ff4e53b03206a746904fa3 2023-05-02 02:55:25,765 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/78fb849dbb85ed1018811c0b9ad29f02/cf1/84b2009af5f3401287c117850ef9fc4b.059f75e2e092770bf7a63b32739cb003 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/78fb849dbb85ed1018811c0b9ad29f02/cf1/84b2009af5f3401287c117850ef9fc4b.059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:25,765 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/f920d56a4f8ae9570632b12801c0f779/cf1/0837374a7f2a4c82bfa97443781ee242.059f75e2e092770bf7a63b32739cb003 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/f920d56a4f8ae9570632b12801c0f779/cf1/0837374a7f2a4c82bfa97443781ee242.059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:25,765 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/e5774ef6b56443f583eeef35a0086dd7 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf1/e5774ef6b56443f583eeef35a0086dd7 2023-05-02 02:55:25,766 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/78fb849dbb85ed1018811c0b9ad29f02/cf1/8f67930f70934c80893ea92e3da97da1.059f75e2e092770bf7a63b32739cb003 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/78fb849dbb85ed1018811c0b9ad29f02/cf1/8f67930f70934c80893ea92e3da97da1.059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:25,766 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/f920d56a4f8ae9570632b12801c0f779/cf1/21a49995122e48a493b6d0a8abb3ff52.059f75e2e092770bf7a63b32739cb003 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/f920d56a4f8ae9570632b12801c0f779/cf1/21a49995122e48a493b6d0a8abb3ff52.059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:25,767 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/78fb849dbb85ed1018811c0b9ad29f02/cf1/b1149ff9cf704fc0b06d4a93d73229dc.059f75e2e092770bf7a63b32739cb003 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/78fb849dbb85ed1018811c0b9ad29f02/cf1/b1149ff9cf704fc0b06d4a93d73229dc.059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:25,768 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/f920d56a4f8ae9570632b12801c0f779/cf1/84b2009af5f3401287c117850ef9fc4b.059f75e2e092770bf7a63b32739cb003 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/f920d56a4f8ae9570632b12801c0f779/cf1/84b2009af5f3401287c117850ef9fc4b.059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:25,768 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/4ddd3ec587eb4d9f9c2d89d53cdc963f to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/4ddd3ec587eb4d9f9c2d89d53cdc963f 2023-05-02 02:55:25,768 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/78fb849dbb85ed1018811c0b9ad29f02/cf1/bc695fd948ff4e53b03206a746904fa3.059f75e2e092770bf7a63b32739cb003 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/78fb849dbb85ed1018811c0b9ad29f02/cf1/bc695fd948ff4e53b03206a746904fa3.059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:25,769 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/f920d56a4f8ae9570632b12801c0f779/cf1/8f67930f70934c80893ea92e3da97da1.059f75e2e092770bf7a63b32739cb003 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/f920d56a4f8ae9570632b12801c0f779/cf1/8f67930f70934c80893ea92e3da97da1.059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:25,769 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/78391fd40923430ba069adcd774c9f4b to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/78391fd40923430ba069adcd774c9f4b 2023-05-02 02:55:25,770 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/78fb849dbb85ed1018811c0b9ad29f02/cf1/e5774ef6b56443f583eeef35a0086dd7.059f75e2e092770bf7a63b32739cb003 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/78fb849dbb85ed1018811c0b9ad29f02/cf1/e5774ef6b56443f583eeef35a0086dd7.059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:25,770 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/f920d56a4f8ae9570632b12801c0f779/cf1/b1149ff9cf704fc0b06d4a93d73229dc.059f75e2e092770bf7a63b32739cb003 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/f920d56a4f8ae9570632b12801c0f779/cf1/b1149ff9cf704fc0b06d4a93d73229dc.059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:25,770 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/902bebca210f45908626f299bace6411 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/902bebca210f45908626f299bace6411 2023-05-02 02:55:25,771 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/9a1be768ec4e4cdbadc55ffdbfe82828 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/9a1be768ec4e4cdbadc55ffdbfe82828 2023-05-02 02:55:25,771 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/f920d56a4f8ae9570632b12801c0f779/cf1/bc695fd948ff4e53b03206a746904fa3.059f75e2e092770bf7a63b32739cb003 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/f920d56a4f8ae9570632b12801c0f779/cf1/bc695fd948ff4e53b03206a746904fa3.059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:25,773 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/b3d0ee38ca69423f9a6877578518bc76 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/b3d0ee38ca69423f9a6877578518bc76 2023-05-02 02:55:25,773 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/f920d56a4f8ae9570632b12801c0f779/cf1/e5774ef6b56443f583eeef35a0086dd7.059f75e2e092770bf7a63b32739cb003 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/f920d56a4f8ae9570632b12801c0f779/cf1/e5774ef6b56443f583eeef35a0086dd7.059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:25,773 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/78fb849dbb85ed1018811c0b9ad29f02/cf2/4ddd3ec587eb4d9f9c2d89d53cdc963f.059f75e2e092770bf7a63b32739cb003 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/78fb849dbb85ed1018811c0b9ad29f02/cf2/4ddd3ec587eb4d9f9c2d89d53cdc963f.059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:25,774 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/b8675e8553444d2bbde903dca91aba1e to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/b8675e8553444d2bbde903dca91aba1e 2023-05-02 02:55:25,775 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/78fb849dbb85ed1018811c0b9ad29f02/cf2/78391fd40923430ba069adcd774c9f4b.059f75e2e092770bf7a63b32739cb003 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/78fb849dbb85ed1018811c0b9ad29f02/cf2/78391fd40923430ba069adcd774c9f4b.059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:25,775 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/b9608c129d224c4fa22a6083bbb53907 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/cf2/b9608c129d224c4fa22a6083bbb53907 2023-05-02 02:55:25,776 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/78fb849dbb85ed1018811c0b9ad29f02/cf2/902bebca210f45908626f299bace6411.059f75e2e092770bf7a63b32739cb003 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/78fb849dbb85ed1018811c0b9ad29f02/cf2/902bebca210f45908626f299bace6411.059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:25,776 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/f920d56a4f8ae9570632b12801c0f779/cf2/4ddd3ec587eb4d9f9c2d89d53cdc963f.059f75e2e092770bf7a63b32739cb003 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/f920d56a4f8ae9570632b12801c0f779/cf2/4ddd3ec587eb4d9f9c2d89d53cdc963f.059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:25,777 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/78fb849dbb85ed1018811c0b9ad29f02/cf2/9a1be768ec4e4cdbadc55ffdbfe82828.059f75e2e092770bf7a63b32739cb003 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/78fb849dbb85ed1018811c0b9ad29f02/cf2/9a1be768ec4e4cdbadc55ffdbfe82828.059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:25,777 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/f920d56a4f8ae9570632b12801c0f779/cf2/78391fd40923430ba069adcd774c9f4b.059f75e2e092770bf7a63b32739cb003 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/f920d56a4f8ae9570632b12801c0f779/cf2/78391fd40923430ba069adcd774c9f4b.059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:25,778 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/recovered.edits/85.seqid to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003/recovered.edits/85.seqid 2023-05-02 02:55:25,778 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/78fb849dbb85ed1018811c0b9ad29f02/cf2/b3d0ee38ca69423f9a6877578518bc76.059f75e2e092770bf7a63b32739cb003 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/78fb849dbb85ed1018811c0b9ad29f02/cf2/b3d0ee38ca69423f9a6877578518bc76.059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:25,779 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:25,779 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/f920d56a4f8ae9570632b12801c0f779/cf2/902bebca210f45908626f299bace6411.059f75e2e092770bf7a63b32739cb003 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/f920d56a4f8ae9570632b12801c0f779/cf2/902bebca210f45908626f299bace6411.059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:25,780 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/78fb849dbb85ed1018811c0b9ad29f02/cf2/b8675e8553444d2bbde903dca91aba1e.059f75e2e092770bf7a63b32739cb003 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/78fb849dbb85ed1018811c0b9ad29f02/cf2/b8675e8553444d2bbde903dca91aba1e.059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:25,780 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/f920d56a4f8ae9570632b12801c0f779/cf2/9a1be768ec4e4cdbadc55ffdbfe82828.059f75e2e092770bf7a63b32739cb003 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/f920d56a4f8ae9570632b12801c0f779/cf2/9a1be768ec4e4cdbadc55ffdbfe82828.059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:25,781 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/78fb849dbb85ed1018811c0b9ad29f02/cf2/b9608c129d224c4fa22a6083bbb53907.059f75e2e092770bf7a63b32739cb003 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/78fb849dbb85ed1018811c0b9ad29f02/cf2/b9608c129d224c4fa22a6083bbb53907.059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:25,781 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/f920d56a4f8ae9570632b12801c0f779/cf2/b3d0ee38ca69423f9a6877578518bc76.059f75e2e092770bf7a63b32739cb003 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/f920d56a4f8ae9570632b12801c0f779/cf2/b3d0ee38ca69423f9a6877578518bc76.059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:25,782 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/f920d56a4f8ae9570632b12801c0f779/cf2/b8675e8553444d2bbde903dca91aba1e.059f75e2e092770bf7a63b32739cb003 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/f920d56a4f8ae9570632b12801c0f779/cf2/b8675e8553444d2bbde903dca91aba1e.059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:25,783 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf1/955625a134714786a0f4faf66d4cb5ca to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf1/955625a134714786a0f4faf66d4cb5ca 2023-05-02 02:55:25,783 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/f920d56a4f8ae9570632b12801c0f779/cf2/b9608c129d224c4fa22a6083bbb53907.059f75e2e092770bf7a63b32739cb003 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/f920d56a4f8ae9570632b12801c0f779/cf2/b9608c129d224c4fa22a6083bbb53907.059f75e2e092770bf7a63b32739cb003 2023-05-02 02:55:25,785 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf2/f53ee6b32b954757a07448983421d641 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/cf2/f53ee6b32b954757a07448983421d641 2023-05-02 02:55:25,786 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/recovered.edits/90.seqid to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779/recovered.edits/90.seqid 2023-05-02 02:55:25,787 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/f920d56a4f8ae9570632b12801c0f779 2023-05-02 02:55:25,787 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/recovered.edits/90.seqid to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02/recovered.edits/90.seqid 2023-05-02 02:55:25,788 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testRecoveryAndDoubleExecution/78fb849dbb85ed1018811c0b9ad29f02 2023-05-02 02:55:25,788 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-05-02 02:55:25,790 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=124, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-05-02 02:55:25,792 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testRecoveryAndDoubleExecution from hbase:meta 2023-05-02 02:55:25,793 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRecoveryAndDoubleExecution' descriptor. 2023-05-02 02:55:25,794 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=124, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-05-02 02:55:25,794 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRecoveryAndDoubleExecution' from region states. 2023-05-02 02:55:25,794 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1682996118490.059f75e2e092770bf7a63b32739cb003.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1682996125794"}]},"ts":"9223372036854775807"} 2023-05-02 02:55:25,794 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1682996120349.78fb849dbb85ed1018811c0b9ad29f02.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1682996125794"}]},"ts":"9223372036854775807"} 2023-05-02 02:55:25,794 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,41,1682996120349.f920d56a4f8ae9570632b12801c0f779.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1682996125794"}]},"ts":"9223372036854775807"} 2023-05-02 02:55:25,796 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-05-02 02:55:25,796 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 059f75e2e092770bf7a63b32739cb003, NAME => 'testRecoveryAndDoubleExecution,,1682996118490.059f75e2e092770bf7a63b32739cb003.', STARTKEY => '', ENDKEY => '', OFFLINE => true, SPLIT => true}, {ENCODED => 78fb849dbb85ed1018811c0b9ad29f02, NAME => 'testRecoveryAndDoubleExecution,,1682996120349.78fb849dbb85ed1018811c0b9ad29f02.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => f920d56a4f8ae9570632b12801c0f779, NAME => 'testRecoveryAndDoubleExecution,41,1682996120349.f920d56a4f8ae9570632b12801c0f779.', STARTKEY => '41', ENDKEY => ''}] 2023-05-02 02:55:25,796 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRecoveryAndDoubleExecution' as deleted. 2023-05-02 02:55:25,796 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1682996125796"}]},"ts":"9223372036854775807"} 2023-05-02 02:55:25,797 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRecoveryAndDoubleExecution state from META 2023-05-02 02:55:25,799 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=124, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-05-02 02:55:25,800 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=124, state=SUCCESS; DeleteTableProcedure table=testRecoveryAndDoubleExecution in 57 msec 2023-05-02 02:55:25,846 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1227): Checking to see if procedure is done pid=124 2023-05-02 02:55:25,846 INFO [Listener at localhost/37119] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRecoveryAndDoubleExecution, procId: 124 completed 2023-05-02 02:55:25,863 INFO [Listener at localhost/37119] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=376 (was 369) - Thread LEAK? -, OpenFileDescriptor=838 (was 744) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=147 (was 155), ProcessCount=170 (was 170), AvailableMemoryMB=13750 (was 13817) 2023-05-02 02:55:25,878 INFO [Listener at localhost/37119] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=376, OpenFileDescriptor=838, MaxFileDescriptor=60000, SystemLoadAverage=147, ProcessCount=170, AvailableMemoryMB=13749 2023-05-02 02:55:25,878 WARN [Listener at localhost/37119] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-02 02:55:25,881 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-05-02 02:55:25,882 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-05-02 02:55:25,883 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(1029): Stored pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionEmptyDaughter 2023-05-02 02:55:25,884 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-05-02 02:55:25,885 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-05-02 02:55:25,886 DEBUG [HFileArchiver-16] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808 2023-05-02 02:55:25,887 DEBUG [HFileArchiver-16] backup.HFileArchiver(153): Directory hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808 empty. 2023-05-02 02:55:25,887 DEBUG [HFileArchiver-16] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808 2023-05-02 02:55:25,887 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-05-02 02:55:25,895 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionEmptyDaughter/.tabledesc/.tableinfo.0000000001 2023-05-02 02:55:25,896 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(7675): creating {ENCODED => d41bae7ce64ef0743e56e5fbc14a3808, NAME => 'testSplitTableRegionEmptyDaughter,,1682996125882.d41bae7ce64ef0743e56e5fbc14a3808.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionEmptyDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp 2023-05-02 02:55:25,903 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegionEmptyDaughter,,1682996125882.d41bae7ce64ef0743e56e5fbc14a3808.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-02 02:55:25,903 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1604): Closing d41bae7ce64ef0743e56e5fbc14a3808, disabling compactions & flushes 2023-05-02 02:55:25,903 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegionEmptyDaughter,,1682996125882.d41bae7ce64ef0743e56e5fbc14a3808. 2023-05-02 02:55:25,903 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1682996125882.d41bae7ce64ef0743e56e5fbc14a3808. 2023-05-02 02:55:25,903 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionEmptyDaughter,,1682996125882.d41bae7ce64ef0743e56e5fbc14a3808. after waiting 0 ms 2023-05-02 02:55:25,903 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionEmptyDaughter,,1682996125882.d41bae7ce64ef0743e56e5fbc14a3808. 2023-05-02 02:55:25,903 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegionEmptyDaughter,,1682996125882.d41bae7ce64ef0743e56e5fbc14a3808. 2023-05-02 02:55:25,903 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1558): Region close journal for d41bae7ce64ef0743e56e5fbc14a3808: 2023-05-02 02:55:25,905 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-05-02 02:55:25,905 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1682996125882.d41bae7ce64ef0743e56e5fbc14a3808.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1682996125905"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682996125905"}]},"ts":"1682996125905"} 2023-05-02 02:55:25,907 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-05-02 02:55:25,908 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-05-02 02:55:25,908 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682996125908"}]},"ts":"1682996125908"} 2023-05-02 02:55:25,909 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLING in hbase:meta 2023-05-02 02:55:25,911 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-05-02 02:55:25,912 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-05-02 02:55:25,912 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-05-02 02:55:25,912 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-05-02 02:55:25,912 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-05-02 02:55:25,912 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=d41bae7ce64ef0743e56e5fbc14a3808, ASSIGN}] 2023-05-02 02:55:25,914 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=d41bae7ce64ef0743e56e5fbc14a3808, ASSIGN 2023-05-02 02:55:25,914 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=d41bae7ce64ef0743e56e5fbc14a3808, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,39081,1682996021579; forceNewPlan=false, retain=false 2023-05-02 02:55:26,064 INFO [jenkins-hbase4:41119] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-05-02 02:55:26,065 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=d41bae7ce64ef0743e56e5fbc14a3808, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:26,066 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1682996125882.d41bae7ce64ef0743e56e5fbc14a3808.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1682996126065"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1682996126065"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682996126065"}]},"ts":"1682996126065"} 2023-05-02 02:55:26,067 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=127, ppid=126, state=RUNNABLE; OpenRegionProcedure d41bae7ce64ef0743e56e5fbc14a3808, server=jenkins-hbase4.apache.org,39081,1682996021579}] 2023-05-02 02:55:26,210 ERROR [BP-376276558-172.31.14.131-1682996016523 heartbeating to localhost/127.0.0.1:38649] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-376276558-172.31.14.131-1682996016523 (Datanode Uuid 2b091f00-29ea-476e-ba4f-8ff3dee54078) service to localhost/127.0.0.1:38649 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-05-02 02:55:26,224 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionEmptyDaughter,,1682996125882.d41bae7ce64ef0743e56e5fbc14a3808. 2023-05-02 02:55:26,224 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => d41bae7ce64ef0743e56e5fbc14a3808, NAME => 'testSplitTableRegionEmptyDaughter,,1682996125882.d41bae7ce64ef0743e56e5fbc14a3808.', STARTKEY => '', ENDKEY => ''} 2023-05-02 02:55:26,225 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-02 02:55:26,225 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter d41bae7ce64ef0743e56e5fbc14a3808 2023-05-02 02:55:26,225 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionEmptyDaughter,,1682996125882.d41bae7ce64ef0743e56e5fbc14a3808.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-02 02:55:26,225 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for d41bae7ce64ef0743e56e5fbc14a3808 2023-05-02 02:55:26,225 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for d41bae7ce64ef0743e56e5fbc14a3808 2023-05-02 02:55:26,226 INFO [StoreOpener-d41bae7ce64ef0743e56e5fbc14a3808-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region d41bae7ce64ef0743e56e5fbc14a3808 2023-05-02 02:55:26,227 DEBUG [StoreOpener-d41bae7ce64ef0743e56e5fbc14a3808-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf1 2023-05-02 02:55:26,227 DEBUG [StoreOpener-d41bae7ce64ef0743e56e5fbc14a3808-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf1 2023-05-02 02:55:26,228 INFO [StoreOpener-d41bae7ce64ef0743e56e5fbc14a3808-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d41bae7ce64ef0743e56e5fbc14a3808 columnFamilyName cf1 2023-05-02 02:55:26,228 INFO [StoreOpener-d41bae7ce64ef0743e56e5fbc14a3808-1] regionserver.HStore(310): Store=d41bae7ce64ef0743e56e5fbc14a3808/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-02 02:55:26,228 INFO [StoreOpener-d41bae7ce64ef0743e56e5fbc14a3808-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region d41bae7ce64ef0743e56e5fbc14a3808 2023-05-02 02:55:26,229 DEBUG [StoreOpener-d41bae7ce64ef0743e56e5fbc14a3808-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf2 2023-05-02 02:55:26,229 DEBUG [StoreOpener-d41bae7ce64ef0743e56e5fbc14a3808-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf2 2023-05-02 02:55:26,230 INFO [StoreOpener-d41bae7ce64ef0743e56e5fbc14a3808-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d41bae7ce64ef0743e56e5fbc14a3808 columnFamilyName cf2 2023-05-02 02:55:26,230 INFO [StoreOpener-d41bae7ce64ef0743e56e5fbc14a3808-1] regionserver.HStore(310): Store=d41bae7ce64ef0743e56e5fbc14a3808/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-02 02:55:26,231 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808 2023-05-02 02:55:26,231 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808 2023-05-02 02:55:26,233 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-02 02:55:26,234 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for d41bae7ce64ef0743e56e5fbc14a3808 2023-05-02 02:55:26,235 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-05-02 02:55:26,236 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened d41bae7ce64ef0743e56e5fbc14a3808; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74373449, jitterRate=0.10825075209140778}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-02 02:55:26,236 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for d41bae7ce64ef0743e56e5fbc14a3808: 2023-05-02 02:55:26,236 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1682996125882.d41bae7ce64ef0743e56e5fbc14a3808., pid=127, masterSystemTime=1682996126218 2023-05-02 02:55:26,238 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1682996125882.d41bae7ce64ef0743e56e5fbc14a3808. 2023-05-02 02:55:26,238 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionEmptyDaughter,,1682996125882.d41bae7ce64ef0743e56e5fbc14a3808. 2023-05-02 02:55:26,238 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=d41bae7ce64ef0743e56e5fbc14a3808, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:26,238 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1682996125882.d41bae7ce64ef0743e56e5fbc14a3808.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1682996126238"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1682996126238"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1682996126238"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682996126238"}]},"ts":"1682996126238"} 2023-05-02 02:55:26,241 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=127, resume processing ppid=126 2023-05-02 02:55:26,241 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=127, ppid=126, state=SUCCESS; OpenRegionProcedure d41bae7ce64ef0743e56e5fbc14a3808, server=jenkins-hbase4.apache.org,39081,1682996021579 in 173 msec 2023-05-02 02:55:26,242 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=126, resume processing ppid=125 2023-05-02 02:55:26,242 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=126, ppid=125, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=d41bae7ce64ef0743e56e5fbc14a3808, ASSIGN in 329 msec 2023-05-02 02:55:26,243 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-05-02 02:55:26,243 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682996126243"}]},"ts":"1682996126243"} 2023-05-02 02:55:26,244 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLED in hbase:meta 2023-05-02 02:55:26,246 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-05-02 02:55:26,247 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=125, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionEmptyDaughter in 364 msec 2023-05-02 02:55:26,391 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-02 02:55:26,393 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,393 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-05-02 02:55:26,393 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:26,393 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-05-02 02:55:26,393 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-05-02 02:55:26,393 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,393 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,395 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:26,395 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:26,395 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:26,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:26,395 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:26,395 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:26,395 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:26,395 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:26,395 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:26,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:26,396 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-02 02:55:26,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:26,396 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:26,396 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,396 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,396 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:26,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:26,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,397 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,397 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,397 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,397 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-02 02:55:26,397 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:26,397 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-02 02:55:26,397 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,397 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,397 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-02 02:55:26,397 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-02 02:55:26,397 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:26,397 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:26,397 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-02 02:55:26,397 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-02 02:55:26,397 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42019,1682996021676' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-02 02:55:26,397 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-02 02:55:26,397 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-02 02:55:26,397 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38925,1682996021633' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-02 02:55:26,398 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-02 02:55:26,398 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-02 02:55:26,398 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1682996125882.d41bae7ce64ef0743e56e5fbc14a3808. 2023-05-02 02:55:26,398 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1682996125882.d41bae7ce64ef0743e56e5fbc14a3808. started... 2023-05-02 02:55:26,399 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d41bae7ce64ef0743e56e5fbc14a3808 2/2 column families, dataSize=120 B heapSize=928 B 2023-05-02 02:55:26,400 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,400 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,400 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,400 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:26,400 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,400 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:26,407 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/.tmp/cf1/4a91258191884da1b8c056d2aa3e968b 2023-05-02 02:55:26,417 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/.tmp/cf2/0a3e2789ee70441192f6dbc5f13b73b4 2023-05-02 02:55:26,420 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/.tmp/cf1/4a91258191884da1b8c056d2aa3e968b as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf1/4a91258191884da1b8c056d2aa3e968b 2023-05-02 02:55:26,424 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf1/4a91258191884da1b8c056d2aa3e968b, entries=2, sequenceid=6, filesize=4.8 K 2023-05-02 02:55:26,424 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/.tmp/cf2/0a3e2789ee70441192f6dbc5f13b73b4 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf2/0a3e2789ee70441192f6dbc5f13b73b4 2023-05-02 02:55:26,427 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf2/0a3e2789ee70441192f6dbc5f13b73b4, entries=2, sequenceid=6, filesize=4.8 K 2023-05-02 02:55:26,428 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for d41bae7ce64ef0743e56e5fbc14a3808 in 29ms, sequenceid=6, compaction requested=false 2023-05-02 02:55:26,428 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionEmptyDaughter' 2023-05-02 02:55:26,428 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d41bae7ce64ef0743e56e5fbc14a3808: 2023-05-02 02:55:26,428 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1682996125882.d41bae7ce64ef0743e56e5fbc14a3808. 2023-05-02 02:55:26,428 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-02 02:55:26,428 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-02 02:55:26,428 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:26,428 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-02 02:55:26,428 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-02 02:55:26,430 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:26,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:26,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:26,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:26,430 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:26,430 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,430 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:26,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:26,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:26,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:26,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:26,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:26,432 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-05-02 02:55:26,432 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4e533667[Count = 0] remaining members to acquire global barrier 2023-05-02 02:55:26,432 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-05-02 02:55:26,432 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,433 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,433 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,433 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,433 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,433 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,433 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:26,433 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,433 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-02 02:55:26,433 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-02 02:55:26,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,433 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-02 02:55:26,433 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-02 02:55:26,433 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,39081,1682996021579' in zk 2023-05-02 02:55:26,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,433 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-02 02:55:26,433 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,38925,1682996021633' in zk 2023-05-02 02:55:26,433 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-02 02:55:26,433 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-02 02:55:26,433 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,42019,1682996021676' in zk 2023-05-02 02:55:26,435 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:26,435 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-02 02:55:26,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:26,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:26,435 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-02 02:55:26,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:26,435 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:26,435 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:26,435 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-02 02:55:26,435 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-02 02:55:26,436 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-02 02:55:26,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:26,436 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:26,436 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-02 02:55:26,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:26,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:26,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:26,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:26,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:26,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:26,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:26,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:26,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,39081,1682996021579': 2023-05-02 02:55:26,438 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-05-02 02:55:26,438 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-05-02 02:55:26,438 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-02 02:55:26,438 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-02 02:55:26,438 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,438 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-02 02:55:26,439 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,439 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:26,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:26,439 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,440 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,440 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,440 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:26,440 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:26,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:26,440 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,440 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,440 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:26,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,440 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:26,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,440 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:26,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:26,440 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:26,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:26,440 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:26,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:26,441 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:26,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:26,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,441 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:26,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:26,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:26,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:26,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:26,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,442 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:26,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:26,443 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:26,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:26,443 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:26,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:26,449 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:26,450 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:26,450 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:26,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:26,450 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:26,450 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:26,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:26,450 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:26,450 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:26,450 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:26,450 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,450 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:26,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:26,450 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:26,450 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-05-02 02:55:26,450 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-02 02:55:26,450 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:26,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:26,450 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:26,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:26,450 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:26,450 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:26,450 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,450 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:26,450 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-05-02 02:55:26,450 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:26,451 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-02 02:55:26,450 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,451 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:26,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:26,451 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:26,451 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:26,451 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,451 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,551 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-02 02:55:26,552 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-02 02:55:26,562 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-02 02:55:26,563 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-05-02 02:55:26,563 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,564 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-05-02 02:55:26,564 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:26,564 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-05-02 02:55:26,564 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-05-02 02:55:26,564 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,564 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,565 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:26,565 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:26,565 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:26,565 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:26,565 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:26,565 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:26,565 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:26,566 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-02 02:55:26,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:26,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:26,565 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:26,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:26,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,566 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,566 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,566 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:26,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:26,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,567 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,567 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:26,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,567 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,567 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-05-02 02:55:26,567 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,567 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-05-02 02:55:26,567 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-02 02:55:26,567 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-05-02 02:55:26,567 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:26,567 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,567 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-02 02:55:26,567 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,567 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-02 02:55:26,567 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-02 02:55:26,567 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-02 02:55:26,567 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-02 02:55:26,567 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:26,567 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:26,567 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-02 02:55:26,567 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-02 02:55:26,567 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-02 02:55:26,567 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-02 02:55:26,567 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42019,1682996021676' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-02 02:55:26,567 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1682996125882.d41bae7ce64ef0743e56e5fbc14a3808. 2023-05-02 02:55:26,567 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38925,1682996021633' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-02 02:55:26,568 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1682996125882.d41bae7ce64ef0743e56e5fbc14a3808. started... 2023-05-02 02:55:26,568 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d41bae7ce64ef0743e56e5fbc14a3808 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-02 02:55:26,569 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,569 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,569 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,569 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:26,569 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,569 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:26,576 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/.tmp/cf1/05ef4a10ee5f4a17bac25ec17d04e9bb 2023-05-02 02:55:26,585 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/.tmp/cf2/3f6de7b2a6364f838124fe62355ae13f 2023-05-02 02:55:26,588 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/.tmp/cf1/05ef4a10ee5f4a17bac25ec17d04e9bb as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf1/05ef4a10ee5f4a17bac25ec17d04e9bb 2023-05-02 02:55:26,592 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf1/05ef4a10ee5f4a17bac25ec17d04e9bb, entries=10, sequenceid=19, filesize=5.0 K 2023-05-02 02:55:26,592 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/.tmp/cf2/3f6de7b2a6364f838124fe62355ae13f as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf2/3f6de7b2a6364f838124fe62355ae13f 2023-05-02 02:55:26,595 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf2/3f6de7b2a6364f838124fe62355ae13f, entries=10, sequenceid=19, filesize=5.0 K 2023-05-02 02:55:26,596 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d41bae7ce64ef0743e56e5fbc14a3808 in 28ms, sequenceid=19, compaction requested=false 2023-05-02 02:55:26,596 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d41bae7ce64ef0743e56e5fbc14a3808: 2023-05-02 02:55:26,596 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1682996125882.d41bae7ce64ef0743e56e5fbc14a3808. 2023-05-02 02:55:26,596 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-02 02:55:26,596 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-02 02:55:26,596 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:26,596 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-02 02:55:26,596 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-02 02:55:26,597 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,597 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:26,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:26,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:26,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:26,598 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,598 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:26,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:26,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:26,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:26,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:26,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:26,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:26,599 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-05-02 02:55:26,599 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@53de1f30[Count = 0] remaining members to acquire global barrier 2023-05-02 02:55:26,599 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-05-02 02:55:26,599 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,601 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,601 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,601 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,601 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,601 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,601 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:26,601 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-02 02:55:26,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,601 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-02 02:55:26,602 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-02 02:55:26,602 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-02 02:55:26,602 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,42019,1682996021676' in zk 2023-05-02 02:55:26,601 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-02 02:55:26,601 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,602 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,38925,1682996021633' in zk 2023-05-02 02:55:26,602 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-02 02:55:26,602 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-02 02:55:26,602 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,39081,1682996021579' in zk 2023-05-02 02:55:26,603 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-02 02:55:26,603 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:26,603 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-02 02:55:26,604 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-02 02:55:26,604 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:26,604 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:26,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:26,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:26,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:26,604 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-02 02:55:26,604 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-02 02:55:26,604 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:26,604 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-02 02:55:26,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:26,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:26,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:26,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:26,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:26,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:26,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:26,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:26,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:26,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,39081,1682996021579': 2023-05-02 02:55:26,606 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-05-02 02:55:26,606 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-05-02 02:55:26,606 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-02 02:55:26,606 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-02 02:55:26,606 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,606 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-02 02:55:26,608 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,608 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,608 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,608 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,608 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,608 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:26,608 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,608 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,608 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:26,608 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:26,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:26,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:26,608 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:26,608 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:26,608 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:26,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:26,608 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:26,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:26,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:26,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:26,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,609 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:26,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,609 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:26,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:26,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:26,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:26,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:26,610 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:26,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:26,611 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:26,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,611 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:26,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:26,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:26,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:26,614 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:26,615 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,615 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:26,615 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:26,615 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:26,615 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-05-02 02:55:26,615 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-02 02:55:26,615 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:26,615 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:26,615 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:26,615 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:26,615 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:26,615 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:26,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:26,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:26,615 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-05-02 02:55:26,615 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:26,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:26,615 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:26,615 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:26,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:26,615 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:26,615 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:26,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:26,615 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:26,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:26,615 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-02 02:55:26,615 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,616 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:26,616 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,616 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:26,616 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:26,616 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,616 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,716 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-02 02:55:26,717 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-02 02:55:26,727 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-02 02:55:26,728 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-05-02 02:55:26,729 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,729 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-05-02 02:55:26,729 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:26,729 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-05-02 02:55:26,729 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-05-02 02:55:26,729 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,729 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,730 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:26,730 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:26,730 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:26,731 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:26,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:26,730 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:26,730 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:26,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:26,731 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:26,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:26,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,731 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:26,731 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-02 02:55:26,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,732 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,732 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,732 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:26,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:26,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:26,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,732 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,732 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,732 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,733 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-05-02 02:55:26,733 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,733 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-05-02 02:55:26,733 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-02 02:55:26,733 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:26,733 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,733 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-02 02:55:26,733 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-02 02:55:26,733 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-05-02 02:55:26,733 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-02 02:55:26,733 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,733 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:26,733 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-02 02:55:26,733 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-02 02:55:26,733 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1682996125882.d41bae7ce64ef0743e56e5fbc14a3808. 2023-05-02 02:55:26,733 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-02 02:55:26,733 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-02 02:55:26,733 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38925,1682996021633' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-02 02:55:26,733 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1682996125882.d41bae7ce64ef0743e56e5fbc14a3808. started... 2023-05-02 02:55:26,733 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:26,733 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-02 02:55:26,733 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-02 02:55:26,733 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42019,1682996021676' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-02 02:55:26,734 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d41bae7ce64ef0743e56e5fbc14a3808 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-02 02:55:26,735 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,735 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,735 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,735 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:26,735 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:26,735 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:26,848 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-05-02 02:55:27,147 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/.tmp/cf1/133b1126a5fa48ca9e2e56e2e5617012 2023-05-02 02:55:27,158 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/.tmp/cf2/6cc34836bb60497cb8baefa422d13bcd 2023-05-02 02:55:27,161 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/.tmp/cf1/133b1126a5fa48ca9e2e56e2e5617012 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf1/133b1126a5fa48ca9e2e56e2e5617012 2023-05-02 02:55:27,164 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf1/133b1126a5fa48ca9e2e56e2e5617012, entries=10, sequenceid=32, filesize=5.0 K 2023-05-02 02:55:27,165 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/.tmp/cf2/6cc34836bb60497cb8baefa422d13bcd as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf2/6cc34836bb60497cb8baefa422d13bcd 2023-05-02 02:55:27,168 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf2/6cc34836bb60497cb8baefa422d13bcd, entries=10, sequenceid=32, filesize=5.0 K 2023-05-02 02:55:27,168 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d41bae7ce64ef0743e56e5fbc14a3808 in 435ms, sequenceid=32, compaction requested=true 2023-05-02 02:55:27,168 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d41bae7ce64ef0743e56e5fbc14a3808: 2023-05-02 02:55:27,168 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1682996125882.d41bae7ce64ef0743e56e5fbc14a3808. 2023-05-02 02:55:27,169 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-02 02:55:27,169 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-02 02:55:27,169 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:27,169 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-02 02:55:27,169 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-02 02:55:27,170 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:27,170 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:27,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:27,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:27,170 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,170 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:27,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:27,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:27,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:27,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:27,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:27,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:27,172 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-05-02 02:55:27,172 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@625c0f03[Count = 0] remaining members to acquire global barrier 2023-05-02 02:55:27,172 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-05-02 02:55:27,172 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,174 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,174 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,174 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,174 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,174 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,174 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-02 02:55:27,174 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-02 02:55:27,174 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,174 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,38925,1682996021633' in zk 2023-05-02 02:55:27,174 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-02 02:55:27,174 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-02 02:55:27,174 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,39081,1682996021579' in zk 2023-05-02 02:55:27,174 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:27,174 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-02 02:55:27,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,174 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-02 02:55:27,174 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-02 02:55:27,175 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,42019,1682996021676' in zk 2023-05-02 02:55:27,181 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-02 02:55:27,181 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:27,181 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-02 02:55:27,182 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:27,182 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-02 02:55:27,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:27,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:27,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:27,182 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:27,182 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-02 02:55:27,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:27,183 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-02 02:55:27,183 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:27,183 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-02 02:55:27,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:27,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:27,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:27,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:27,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:27,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:27,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:27,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:27,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,39081,1682996021579': 2023-05-02 02:55:27,186 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-05-02 02:55:27,186 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-05-02 02:55:27,186 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-02 02:55:27,186 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-02 02:55:27,186 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,186 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-02 02:55:27,187 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,187 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,187 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,187 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:27,187 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,187 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:27,187 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,187 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,187 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:27,187 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,188 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:27,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:27,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:27,188 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:27,188 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:27,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:27,188 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:27,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:27,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:27,188 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:27,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:27,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,189 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:27,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:27,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:27,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:27,190 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:27,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:27,190 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:27,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:27,191 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:27,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:27,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:27,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:27,193 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:27,193 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:27,193 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:27,193 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,193 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:27,193 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:27,193 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:27,193 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:27,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:27,193 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:27,193 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:27,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:27,193 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:27,193 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:27,193 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:27,194 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:27,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:27,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:27,194 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,194 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:27,194 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:27,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:27,194 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,194 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:27,194 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:27,194 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,194 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,194 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:27,194 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-05-02 02:55:27,194 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-02 02:55:27,194 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:27,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:27,195 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-05-02 02:55:27,195 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-02 02:55:27,295 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-02 02:55:27,296 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-02 02:55:27,307 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-02 02:55:27,308 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-05-02 02:55:27,308 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,308 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-05-02 02:55:27,308 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:27,308 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-05-02 02:55:27,308 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-05-02 02:55:27,309 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,309 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,310 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:27,310 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:27,310 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:27,310 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:27,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:27,310 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:27,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:27,310 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:27,310 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:27,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:27,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,310 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:27,310 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-02 02:55:27,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,310 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,311 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,311 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:27,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,311 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:27,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:27,311 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-05-02 02:55:27,311 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,311 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,311 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-02 02:55:27,311 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:27,311 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-05-02 02:55:27,311 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,312 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,311 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-02 02:55:27,312 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-02 02:55:27,312 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42019,1682996021676' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-02 02:55:27,312 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-02 02:55:27,312 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:27,312 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-02 02:55:27,312 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-05-02 02:55:27,312 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-02 02:55:27,312 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,312 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38925,1682996021633' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-02 02:55:27,312 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-02 02:55:27,312 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:27,312 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-02 02:55:27,312 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-02 02:55:27,312 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-02 02:55:27,312 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1682996125882.d41bae7ce64ef0743e56e5fbc14a3808. 2023-05-02 02:55:27,312 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1682996125882.d41bae7ce64ef0743e56e5fbc14a3808. started... 2023-05-02 02:55:27,313 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d41bae7ce64ef0743e56e5fbc14a3808 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-02 02:55:27,315 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,315 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,315 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,315 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:27,315 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,315 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:27,325 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/.tmp/cf1/2c99538e0b1141f192d62203b1188b2a 2023-05-02 02:55:27,341 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/.tmp/cf2/202e8eb538bb4f4aac7238dd9066a205 2023-05-02 02:55:27,345 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/.tmp/cf1/2c99538e0b1141f192d62203b1188b2a as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf1/2c99538e0b1141f192d62203b1188b2a 2023-05-02 02:55:27,349 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf1/2c99538e0b1141f192d62203b1188b2a, entries=10, sequenceid=45, filesize=5.0 K 2023-05-02 02:55:27,350 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/.tmp/cf2/202e8eb538bb4f4aac7238dd9066a205 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf2/202e8eb538bb4f4aac7238dd9066a205 2023-05-02 02:55:27,354 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf2/202e8eb538bb4f4aac7238dd9066a205, entries=10, sequenceid=45, filesize=5.0 K 2023-05-02 02:55:27,355 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d41bae7ce64ef0743e56e5fbc14a3808 in 43ms, sequenceid=45, compaction requested=true 2023-05-02 02:55:27,355 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d41bae7ce64ef0743e56e5fbc14a3808: 2023-05-02 02:55:27,355 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1682996125882.d41bae7ce64ef0743e56e5fbc14a3808. 2023-05-02 02:55:27,355 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-02 02:55:27,355 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-02 02:55:27,355 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:27,355 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-02 02:55:27,355 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-02 02:55:27,357 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:27,357 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:27,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:27,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:27,357 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,357 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:27,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:27,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:27,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:27,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:27,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:27,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:27,359 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-05-02 02:55:27,359 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@30dadd7[Count = 0] remaining members to acquire global barrier 2023-05-02 02:55:27,359 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-05-02 02:55:27,359 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,360 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,360 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,360 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,360 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,360 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,360 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:27,360 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,360 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-02 02:55:27,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,360 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-02 02:55:27,360 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-02 02:55:27,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,360 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-02 02:55:27,360 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-02 02:55:27,361 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,42019,1682996021676' in zk 2023-05-02 02:55:27,361 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-02 02:55:27,361 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-02 02:55:27,361 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,39081,1682996021579' in zk 2023-05-02 02:55:27,361 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,38925,1682996021633' in zk 2023-05-02 02:55:27,363 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-02 02:55:27,363 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-02 02:55:27,363 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:27,363 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:27,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:27,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:27,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:27,363 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:27,363 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-02 02:55:27,363 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-02 02:55:27,364 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-02 02:55:27,364 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:27,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:27,364 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-02 02:55:27,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:27,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:27,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:27,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:27,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:27,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:27,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:27,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:27,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,39081,1682996021579': 2023-05-02 02:55:27,367 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-05-02 02:55:27,367 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-05-02 02:55:27,367 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-02 02:55:27,367 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-02 02:55:27,367 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,367 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-02 02:55:27,368 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,368 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,368 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:27,368 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,368 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,368 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,368 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,368 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:27,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:27,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:27,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,368 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,368 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:27,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,369 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:27,369 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:27,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:27,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:27,369 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:27,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:27,369 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:27,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:27,369 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:27,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,370 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:27,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:27,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:27,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:27,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:27,371 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:27,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:27,372 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:27,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,372 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:27,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:27,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:27,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:27,377 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:27,377 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:27,377 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:27,377 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:27,377 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:27,377 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-05-02 02:55:27,377 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-02 02:55:27,377 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:27,377 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,378 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:27,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:27,377 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:27,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:27,378 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:27,378 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:27,378 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:27,378 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-05-02 02:55:27,378 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:27,378 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-02 02:55:27,378 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:27,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:27,378 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:27,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:27,378 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:27,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:27,378 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,378 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:27,378 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,378 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:27,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:27,378 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:27,379 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:27,379 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,379 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,478 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-02 02:55:27,479 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-02 02:55:27,489 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-02 02:55:27,490 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-05-02 02:55:27,490 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,491 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-05-02 02:55:27,491 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:27,491 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-05-02 02:55:27,491 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-05-02 02:55:27,491 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,491 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,492 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:27,492 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:27,492 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:27,492 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:27,492 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:27,492 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:27,492 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:27,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:27,492 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:27,492 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-02 02:55:27,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:27,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:27,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,493 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,493 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,493 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:27,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,493 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:27,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:27,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,493 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,494 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,494 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-05-02 02:55:27,494 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,494 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-05-02 02:55:27,494 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-02 02:55:27,494 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-05-02 02:55:27,494 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:27,494 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,494 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,494 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-02 02:55:27,494 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-02 02:55:27,494 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:27,494 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-02 02:55:27,494 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-02 02:55:27,494 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-02 02:55:27,494 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-02 02:55:27,494 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38925,1682996021633' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-02 02:55:27,494 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42019,1682996021676' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-02 02:55:27,494 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:27,494 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-02 02:55:27,494 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-02 02:55:27,494 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-02 02:55:27,494 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1682996125882.d41bae7ce64ef0743e56e5fbc14a3808. 2023-05-02 02:55:27,494 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1682996125882.d41bae7ce64ef0743e56e5fbc14a3808. started... 2023-05-02 02:55:27,495 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d41bae7ce64ef0743e56e5fbc14a3808 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-02 02:55:27,495 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,496 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,496 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,496 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:27,496 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,496 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:27,503 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/.tmp/cf1/fdc47bf1839746f685697b19b502c6e1 2023-05-02 02:55:27,513 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/.tmp/cf2/7bdb429f97934894a104bf928b86330d 2023-05-02 02:55:27,517 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/.tmp/cf1/fdc47bf1839746f685697b19b502c6e1 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf1/fdc47bf1839746f685697b19b502c6e1 2023-05-02 02:55:27,520 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf1/fdc47bf1839746f685697b19b502c6e1, entries=10, sequenceid=58, filesize=5.0 K 2023-05-02 02:55:27,521 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/.tmp/cf2/7bdb429f97934894a104bf928b86330d as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf2/7bdb429f97934894a104bf928b86330d 2023-05-02 02:55:27,524 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf2/7bdb429f97934894a104bf928b86330d, entries=10, sequenceid=58, filesize=5.0 K 2023-05-02 02:55:27,525 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d41bae7ce64ef0743e56e5fbc14a3808 in 30ms, sequenceid=58, compaction requested=true 2023-05-02 02:55:27,525 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d41bae7ce64ef0743e56e5fbc14a3808: 2023-05-02 02:55:27,525 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1682996125882.d41bae7ce64ef0743e56e5fbc14a3808. 2023-05-02 02:55:27,525 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-02 02:55:27,525 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-02 02:55:27,525 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:27,525 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-02 02:55:27,525 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-02 02:55:27,526 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:27,526 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:27,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:27,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:27,527 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,527 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:27,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:27,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:27,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:27,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:27,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:27,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:27,528 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-05-02 02:55:27,528 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@54ba53df[Count = 0] remaining members to acquire global barrier 2023-05-02 02:55:27,528 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-05-02 02:55:27,528 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,531 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,531 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,531 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,531 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,531 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,531 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-02 02:55:27,531 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-02 02:55:27,531 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-02 02:55:27,531 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-02 02:55:27,531 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,531 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:27,531 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-02 02:55:27,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,531 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,42019,1682996021676' in zk 2023-05-02 02:55:27,531 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-02 02:55:27,531 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-02 02:55:27,531 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,39081,1682996021579' in zk 2023-05-02 02:55:27,531 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,38925,1682996021633' in zk 2023-05-02 02:55:27,533 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-02 02:55:27,533 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:27,533 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-02 02:55:27,533 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-02 02:55:27,533 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-02 02:55:27,533 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:27,533 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:27,533 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-02 02:55:27,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:27,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:27,533 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:27,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:27,533 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-02 02:55:27,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:27,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:27,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:27,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:27,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:27,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:27,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:27,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:27,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:27,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,39081,1682996021579': 2023-05-02 02:55:27,536 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-05-02 02:55:27,536 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-05-02 02:55:27,536 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-02 02:55:27,536 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-02 02:55:27,536 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,536 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-02 02:55:27,538 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,538 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,538 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:27,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:27,538 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,538 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,538 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,538 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,538 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:27,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,538 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:27,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,538 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:27,538 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:27,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:27,539 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:27,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:27,539 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:27,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:27,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,539 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:27,539 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:27,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:27,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,539 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:27,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:27,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:27,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:27,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:27,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:27,541 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:27,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,541 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:27,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:27,541 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:27,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:27,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:27,544 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:27,544 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:27,544 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:27,544 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:27,544 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,544 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:27,544 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:27,544 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:27,544 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:27,544 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:27,544 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:27,544 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:27,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:27,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:27,544 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,544 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-05-02 02:55:27,545 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-02 02:55:27,544 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:27,545 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:27,545 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:27,544 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:27,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:27,545 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-05-02 02:55:27,545 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-02 02:55:27,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:27,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:27,545 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:27,545 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,545 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:27,545 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:27,545 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,545 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,545 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:27,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:27,645 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-02 02:55:27,646 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-02 02:55:27,656 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-02 02:55:27,657 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-05-02 02:55:27,657 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,657 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-05-02 02:55:27,657 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:27,657 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-05-02 02:55:27,657 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-05-02 02:55:27,657 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,657 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,658 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:27,658 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:27,659 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:27,659 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:27,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:27,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:27,658 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:27,658 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:27,659 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:27,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:27,659 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:27,659 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-02 02:55:27,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,659 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,660 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,660 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:27,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,660 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:27,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:27,660 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-05-02 02:55:27,660 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,660 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,660 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-02 02:55:27,660 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:27,660 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-05-02 02:55:27,660 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,660 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,660 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-02 02:55:27,660 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-02 02:55:27,660 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:27,661 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-02 02:55:27,661 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-02 02:55:27,661 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42019,1682996021676' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-02 02:55:27,661 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-05-02 02:55:27,661 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-02 02:55:27,661 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-02 02:55:27,661 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,661 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1682996125882.d41bae7ce64ef0743e56e5fbc14a3808. 2023-05-02 02:55:27,661 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-02 02:55:27,661 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:27,661 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1682996125882.d41bae7ce64ef0743e56e5fbc14a3808. started... 2023-05-02 02:55:27,661 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-02 02:55:27,661 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-02 02:55:27,661 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38925,1682996021633' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-02 02:55:27,661 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d41bae7ce64ef0743e56e5fbc14a3808 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-02 02:55:27,662 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,663 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,663 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,663 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:27,663 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,663 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:27,671 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/.tmp/cf1/39192fe710b14e58949a1a029ab4fb87 2023-05-02 02:55:27,681 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/.tmp/cf2/a248d1981ba743d29600881f0ac53928 2023-05-02 02:55:27,684 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/.tmp/cf1/39192fe710b14e58949a1a029ab4fb87 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf1/39192fe710b14e58949a1a029ab4fb87 2023-05-02 02:55:27,687 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf1/39192fe710b14e58949a1a029ab4fb87, entries=10, sequenceid=71, filesize=5.0 K 2023-05-02 02:55:27,688 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/.tmp/cf2/a248d1981ba743d29600881f0ac53928 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf2/a248d1981ba743d29600881f0ac53928 2023-05-02 02:55:27,691 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf2/a248d1981ba743d29600881f0ac53928, entries=10, sequenceid=71, filesize=5.0 K 2023-05-02 02:55:27,692 INFO [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d41bae7ce64ef0743e56e5fbc14a3808 in 31ms, sequenceid=71, compaction requested=true 2023-05-02 02:55:27,692 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d41bae7ce64ef0743e56e5fbc14a3808: 2023-05-02 02:55:27,692 DEBUG [rs(jenkins-hbase4.apache.org,39081,1682996021579)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1682996125882.d41bae7ce64ef0743e56e5fbc14a3808. 2023-05-02 02:55:27,692 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-02 02:55:27,692 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-02 02:55:27,692 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:27,692 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-02 02:55:27,692 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-02 02:55:27,695 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:27,695 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:27,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:27,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:27,695 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,695 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:27,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:27,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:27,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:27,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:27,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:27,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:27,697 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-05-02 02:55:27,697 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@773ef1e7[Count = 0] remaining members to acquire global barrier 2023-05-02 02:55:27,697 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-05-02 02:55:27,697 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,700 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,700 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,700 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,700 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,700 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,700 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-02 02:55:27,700 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:27,700 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-02 02:55:27,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,700 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-02 02:55:27,700 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-02 02:55:27,700 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-02 02:55:27,701 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,38925,1682996021633' in zk 2023-05-02 02:55:27,700 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,700 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,39081,1682996021579' in zk 2023-05-02 02:55:27,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,701 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-02 02:55:27,701 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-02 02:55:27,701 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,42019,1682996021676' in zk 2023-05-02 02:55:27,702 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-02 02:55:27,702 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:27,702 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-02 02:55:27,703 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:27,703 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-02 02:55:27,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:27,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:27,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:27,703 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-02 02:55:27,703 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:27,703 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:27,703 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-02 02:55:27,704 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-02 02:55:27,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:27,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:27,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:27,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:27,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:27,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:27,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:27,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:27,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:27,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,39081,1682996021579': 2023-05-02 02:55:27,706 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-05-02 02:55:27,706 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-05-02 02:55:27,706 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-02 02:55:27,706 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-02 02:55:27,706 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,706 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-02 02:55:27,708 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,708 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,708 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,708 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:27,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:27,708 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,708 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,708 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:27,708 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:27,708 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,708 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:27,708 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:27,708 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:27,708 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:27,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:27,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:27,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:27,708 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:27,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:27,709 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:27,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,709 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:27,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:27,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:27,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:27,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:27,710 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:27,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:27,711 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:27,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,711 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:27,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:27,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:27,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:27,713 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:27,713 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,713 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:27,713 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:27,713 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:27,713 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:27,713 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:27,713 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-05-02 02:55:27,713 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-02 02:55:27,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:27,713 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:27,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:27,713 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:27,713 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:27,713 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:27,714 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:27,714 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-05-02 02:55:27,714 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:27,714 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:27,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:27,714 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,714 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:27,714 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:27,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:27,714 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:27,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:27,714 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:27,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:27,714 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-02 02:55:27,714 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,714 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:27,714 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:27,714 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,714 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-02 02:55:27,814 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-02 02:55:27,815 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-02 02:55:27,823 DEBUG [Listener at localhost/37119] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:27,823 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(1029): Stored pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=d41bae7ce64ef0743e56e5fbc14a3808, daughterA=d317f7d995153dcea00ba9ce59b06174, daughterB=8aefa099a2e821000b0b58916b550287 2023-05-02 02:55:27,824 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=d41bae7ce64ef0743e56e5fbc14a3808, daughterA=d317f7d995153dcea00ba9ce59b06174, daughterB=8aefa099a2e821000b0b58916b550287 2023-05-02 02:55:27,825 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=d41bae7ce64ef0743e56e5fbc14a3808, daughterA=d317f7d995153dcea00ba9ce59b06174, daughterB=8aefa099a2e821000b0b58916b550287 2023-05-02 02:55:27,825 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=d41bae7ce64ef0743e56e5fbc14a3808, daughterA=d317f7d995153dcea00ba9ce59b06174, daughterB=8aefa099a2e821000b0b58916b550287 2023-05-02 02:55:27,826 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=d41bae7ce64ef0743e56e5fbc14a3808, UNASSIGN}] 2023-05-02 02:55:27,827 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=d41bae7ce64ef0743e56e5fbc14a3808, UNASSIGN 2023-05-02 02:55:27,827 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=d41bae7ce64ef0743e56e5fbc14a3808, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:27,827 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1682996125882.d41bae7ce64ef0743e56e5fbc14a3808.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1682996127827"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1682996127827"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682996127827"}]},"ts":"1682996127827"} 2023-05-02 02:55:27,828 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=130, ppid=129, state=RUNNABLE; CloseRegionProcedure d41bae7ce64ef0743e56e5fbc14a3808, server=jenkins-hbase4.apache.org,39081,1682996021579}] 2023-05-02 02:55:27,980 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close d41bae7ce64ef0743e56e5fbc14a3808 2023-05-02 02:55:27,980 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing d41bae7ce64ef0743e56e5fbc14a3808, disabling compactions & flushes 2023-05-02 02:55:27,980 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionEmptyDaughter,,1682996125882.d41bae7ce64ef0743e56e5fbc14a3808. 2023-05-02 02:55:27,980 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1682996125882.d41bae7ce64ef0743e56e5fbc14a3808. 2023-05-02 02:55:27,980 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionEmptyDaughter,,1682996125882.d41bae7ce64ef0743e56e5fbc14a3808. after waiting 0 ms 2023-05-02 02:55:27,980 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionEmptyDaughter,,1682996125882.d41bae7ce64ef0743e56e5fbc14a3808. 2023-05-02 02:55:27,980 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing d41bae7ce64ef0743e56e5fbc14a3808 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-05-02 02:55:27,989 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/.tmp/cf1/25d03ac30c914e069f0bfa4f127e2574 2023-05-02 02:55:28,406 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/.tmp/cf2/dbff77c7426e41e5b02bac811d6395c6 2023-05-02 02:55:28,409 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/.tmp/cf1/25d03ac30c914e069f0bfa4f127e2574 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf1/25d03ac30c914e069f0bfa4f127e2574 2023-05-02 02:55:28,412 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf1/25d03ac30c914e069f0bfa4f127e2574, entries=8, sequenceid=82, filesize=5.0 K 2023-05-02 02:55:28,413 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/.tmp/cf2/dbff77c7426e41e5b02bac811d6395c6 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf2/dbff77c7426e41e5b02bac811d6395c6 2023-05-02 02:55:28,416 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf2/dbff77c7426e41e5b02bac811d6395c6, entries=8, sequenceid=82, filesize=5.0 K 2023-05-02 02:55:28,417 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for d41bae7ce64ef0743e56e5fbc14a3808 in 437ms, sequenceid=82, compaction requested=true 2023-05-02 02:55:28,421 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-05-02 02:55:28,421 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-02 02:55:28,422 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionEmptyDaughter,,1682996125882.d41bae7ce64ef0743e56e5fbc14a3808. 2023-05-02 02:55:28,422 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for d41bae7ce64ef0743e56e5fbc14a3808: 2023-05-02 02:55:28,424 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed d41bae7ce64ef0743e56e5fbc14a3808 2023-05-02 02:55:28,424 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=d41bae7ce64ef0743e56e5fbc14a3808, regionState=CLOSED 2023-05-02 02:55:28,424 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1682996125882.d41bae7ce64ef0743e56e5fbc14a3808.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1682996128424"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682996128424"}]},"ts":"1682996128424"} 2023-05-02 02:55:28,426 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=130, resume processing ppid=129 2023-05-02 02:55:28,426 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=130, ppid=129, state=SUCCESS; CloseRegionProcedure d41bae7ce64ef0743e56e5fbc14a3808, server=jenkins-hbase4.apache.org,39081,1682996021579 in 597 msec 2023-05-02 02:55:28,427 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=129, resume processing ppid=128 2023-05-02 02:55:28,427 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=129, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=d41bae7ce64ef0743e56e5fbc14a3808, UNASSIGN in 600 msec 2023-05-02 02:55:28,437 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=128 splitting 14 storefiles, region=d41bae7ce64ef0743e56e5fbc14a3808, threads=14 2023-05-02 02:55:28,438 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf2/0a3e2789ee70441192f6dbc5f13b73b4 for region: d41bae7ce64ef0743e56e5fbc14a3808 2023-05-02 02:55:28,438 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf2/202e8eb538bb4f4aac7238dd9066a205 for region: d41bae7ce64ef0743e56e5fbc14a3808 2023-05-02 02:55:28,438 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf2/3f6de7b2a6364f838124fe62355ae13f for region: d41bae7ce64ef0743e56e5fbc14a3808 2023-05-02 02:55:28,444 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf2/6cc34836bb60497cb8baefa422d13bcd for region: d41bae7ce64ef0743e56e5fbc14a3808 2023-05-02 02:55:28,444 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf2/7bdb429f97934894a104bf928b86330d for region: d41bae7ce64ef0743e56e5fbc14a3808 2023-05-02 02:55:28,444 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf2/a248d1981ba743d29600881f0ac53928 for region: d41bae7ce64ef0743e56e5fbc14a3808 2023-05-02 02:55:28,444 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf2/dbff77c7426e41e5b02bac811d6395c6 for region: d41bae7ce64ef0743e56e5fbc14a3808 2023-05-02 02:55:28,444 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf1/133b1126a5fa48ca9e2e56e2e5617012 for region: d41bae7ce64ef0743e56e5fbc14a3808 2023-05-02 02:55:28,444 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf1/05ef4a10ee5f4a17bac25ec17d04e9bb for region: d41bae7ce64ef0743e56e5fbc14a3808 2023-05-02 02:55:28,444 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf1/25d03ac30c914e069f0bfa4f127e2574 for region: d41bae7ce64ef0743e56e5fbc14a3808 2023-05-02 02:55:28,444 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf1/2c99538e0b1141f192d62203b1188b2a for region: d41bae7ce64ef0743e56e5fbc14a3808 2023-05-02 02:55:28,445 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf1/39192fe710b14e58949a1a029ab4fb87 for region: d41bae7ce64ef0743e56e5fbc14a3808 2023-05-02 02:55:28,445 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf1/4a91258191884da1b8c056d2aa3e968b for region: d41bae7ce64ef0743e56e5fbc14a3808 2023-05-02 02:55:28,445 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf1/fdc47bf1839746f685697b19b502c6e1 for region: d41bae7ce64ef0743e56e5fbc14a3808 2023-05-02 02:55:28,448 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf2/7bdb429f97934894a104bf928b86330d, top=false 2023-05-02 02:55:28,449 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf1/133b1126a5fa48ca9e2e56e2e5617012, top=false 2023-05-02 02:55:28,449 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf1/05ef4a10ee5f4a17bac25ec17d04e9bb, top=false 2023-05-02 02:55:28,450 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf2/0a3e2789ee70441192f6dbc5f13b73b4, top=false 2023-05-02 02:55:28,452 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf1/25d03ac30c914e069f0bfa4f127e2574, top=false 2023-05-02 02:55:28,452 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf1/2c99538e0b1141f192d62203b1188b2a, top=false 2023-05-02 02:55:28,453 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf2/a248d1981ba743d29600881f0ac53928, top=false 2023-05-02 02:55:28,455 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf2/3f6de7b2a6364f838124fe62355ae13f, top=false 2023-05-02 02:55:28,455 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf2/6cc34836bb60497cb8baefa422d13bcd, top=false 2023-05-02 02:55:28,455 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf1/39192fe710b14e58949a1a029ab4fb87, top=false 2023-05-02 02:55:28,457 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf2/202e8eb538bb4f4aac7238dd9066a205, top=false 2023-05-02 02:55:28,457 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf1/4a91258191884da1b8c056d2aa3e968b, top=false 2023-05-02 02:55:28,457 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf1/fdc47bf1839746f685697b19b502c6e1, top=false 2023-05-02 02:55:28,459 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf2/dbff77c7426e41e5b02bac811d6395c6, top=false 2023-05-02 02:55:28,468 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/.splits/d317f7d995153dcea00ba9ce59b06174/cf1/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-4a91258191884da1b8c056d2aa3e968b for child: d317f7d995153dcea00ba9ce59b06174, parent: d41bae7ce64ef0743e56e5fbc14a3808 2023-05-02 02:55:28,469 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/.splits/d317f7d995153dcea00ba9ce59b06174/cf1/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-25d03ac30c914e069f0bfa4f127e2574 for child: d317f7d995153dcea00ba9ce59b06174, parent: d41bae7ce64ef0743e56e5fbc14a3808 2023-05-02 02:55:28,469 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/.splits/d317f7d995153dcea00ba9ce59b06174/cf1/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-05ef4a10ee5f4a17bac25ec17d04e9bb for child: d317f7d995153dcea00ba9ce59b06174, parent: d41bae7ce64ef0743e56e5fbc14a3808 2023-05-02 02:55:28,471 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/.splits/d317f7d995153dcea00ba9ce59b06174/cf2/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-3f6de7b2a6364f838124fe62355ae13f for child: d317f7d995153dcea00ba9ce59b06174, parent: d41bae7ce64ef0743e56e5fbc14a3808 2023-05-02 02:55:28,473 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf1/25d03ac30c914e069f0bfa4f127e2574 for region: d41bae7ce64ef0743e56e5fbc14a3808 2023-05-02 02:55:28,473 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf1/4a91258191884da1b8c056d2aa3e968b for region: d41bae7ce64ef0743e56e5fbc14a3808 2023-05-02 02:55:28,474 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf1/05ef4a10ee5f4a17bac25ec17d04e9bb for region: d41bae7ce64ef0743e56e5fbc14a3808 2023-05-02 02:55:28,475 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf2/3f6de7b2a6364f838124fe62355ae13f for region: d41bae7ce64ef0743e56e5fbc14a3808 2023-05-02 02:55:28,483 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/.splits/d317f7d995153dcea00ba9ce59b06174/cf2/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-7bdb429f97934894a104bf928b86330d for child: d317f7d995153dcea00ba9ce59b06174, parent: d41bae7ce64ef0743e56e5fbc14a3808 2023-05-02 02:55:28,487 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf2/7bdb429f97934894a104bf928b86330d for region: d41bae7ce64ef0743e56e5fbc14a3808 2023-05-02 02:55:28,490 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/.splits/d317f7d995153dcea00ba9ce59b06174/cf1/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-fdc47bf1839746f685697b19b502c6e1 for child: d317f7d995153dcea00ba9ce59b06174, parent: d41bae7ce64ef0743e56e5fbc14a3808 2023-05-02 02:55:28,490 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/.splits/d317f7d995153dcea00ba9ce59b06174/cf2/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-202e8eb538bb4f4aac7238dd9066a205 for child: d317f7d995153dcea00ba9ce59b06174, parent: d41bae7ce64ef0743e56e5fbc14a3808 2023-05-02 02:55:28,490 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/.splits/d317f7d995153dcea00ba9ce59b06174/cf1/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-133b1126a5fa48ca9e2e56e2e5617012 for child: d317f7d995153dcea00ba9ce59b06174, parent: d41bae7ce64ef0743e56e5fbc14a3808 2023-05-02 02:55:28,492 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/.splits/d317f7d995153dcea00ba9ce59b06174/cf2/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-a248d1981ba743d29600881f0ac53928 for child: d317f7d995153dcea00ba9ce59b06174, parent: d41bae7ce64ef0743e56e5fbc14a3808 2023-05-02 02:55:28,497 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/.splits/d317f7d995153dcea00ba9ce59b06174/cf1/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-39192fe710b14e58949a1a029ab4fb87 for child: d317f7d995153dcea00ba9ce59b06174, parent: d41bae7ce64ef0743e56e5fbc14a3808 2023-05-02 02:55:28,497 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/.splits/d317f7d995153dcea00ba9ce59b06174/cf2/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-0a3e2789ee70441192f6dbc5f13b73b4 for child: d317f7d995153dcea00ba9ce59b06174, parent: d41bae7ce64ef0743e56e5fbc14a3808 2023-05-02 02:55:28,500 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf1/39192fe710b14e58949a1a029ab4fb87 for region: d41bae7ce64ef0743e56e5fbc14a3808 2023-05-02 02:55:28,501 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf2/202e8eb538bb4f4aac7238dd9066a205 for region: d41bae7ce64ef0743e56e5fbc14a3808 2023-05-02 02:55:28,501 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf2/0a3e2789ee70441192f6dbc5f13b73b4 for region: d41bae7ce64ef0743e56e5fbc14a3808 2023-05-02 02:55:28,501 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf1/133b1126a5fa48ca9e2e56e2e5617012 for region: d41bae7ce64ef0743e56e5fbc14a3808 2023-05-02 02:55:28,502 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf2/a248d1981ba743d29600881f0ac53928 for region: d41bae7ce64ef0743e56e5fbc14a3808 2023-05-02 02:55:28,504 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/.splits/d317f7d995153dcea00ba9ce59b06174/cf2/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-6cc34836bb60497cb8baefa422d13bcd for child: d317f7d995153dcea00ba9ce59b06174, parent: d41bae7ce64ef0743e56e5fbc14a3808 2023-05-02 02:55:28,504 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf1/fdc47bf1839746f685697b19b502c6e1 for region: d41bae7ce64ef0743e56e5fbc14a3808 2023-05-02 02:55:28,505 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/.splits/d317f7d995153dcea00ba9ce59b06174/cf2/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-dbff77c7426e41e5b02bac811d6395c6 for child: d317f7d995153dcea00ba9ce59b06174, parent: d41bae7ce64ef0743e56e5fbc14a3808 2023-05-02 02:55:28,505 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/.splits/d317f7d995153dcea00ba9ce59b06174/cf1/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-2c99538e0b1141f192d62203b1188b2a for child: d317f7d995153dcea00ba9ce59b06174, parent: d41bae7ce64ef0743e56e5fbc14a3808 2023-05-02 02:55:28,507 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf2/6cc34836bb60497cb8baefa422d13bcd for region: d41bae7ce64ef0743e56e5fbc14a3808 2023-05-02 02:55:28,507 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf1/2c99538e0b1141f192d62203b1188b2a for region: d41bae7ce64ef0743e56e5fbc14a3808 2023-05-02 02:55:28,508 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf2/dbff77c7426e41e5b02bac811d6395c6 for region: d41bae7ce64ef0743e56e5fbc14a3808 2023-05-02 02:55:28,508 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=128 split storefiles for region d41bae7ce64ef0743e56e5fbc14a3808 Daughter A: 14 storefiles, Daughter B: 0 storefiles. 2023-05-02 02:55:28,537 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-05-02 02:55:28,539 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/8aefa099a2e821000b0b58916b550287/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-05-02 02:55:28,540 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1682996125882.d41bae7ce64ef0743e56e5fbc14a3808.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1682996128540"},{"qualifier":"splitA","vlen":69,"tag":[],"timestamp":"1682996128540"},{"qualifier":"splitB","vlen":69,"tag":[],"timestamp":"1682996128540"}]},"ts":"1682996128540"} 2023-05-02 02:55:28,540 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1682996127823.d317f7d995153dcea00ba9ce59b06174.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1682996128540"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682996128540"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682996128540"}]},"ts":"1682996128540"} 2023-05-02 02:55:28,540 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1682996127823.8aefa099a2e821000b0b58916b550287.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1682996128540"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682996128540"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682996128540"}]},"ts":"1682996128540"} 2023-05-02 02:55:28,543 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=d317f7d995153dcea00ba9ce59b06174, ASSIGN}, {pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=8aefa099a2e821000b0b58916b550287, ASSIGN}] 2023-05-02 02:55:28,543 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=8aefa099a2e821000b0b58916b550287, ASSIGN 2023-05-02 02:55:28,544 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=8aefa099a2e821000b0b58916b550287, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,39081,1682996021579; forceNewPlan=false, retain=false 2023-05-02 02:55:28,544 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=d317f7d995153dcea00ba9ce59b06174, ASSIGN 2023-05-02 02:55:28,545 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=d317f7d995153dcea00ba9ce59b06174, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,39081,1682996021579; forceNewPlan=false, retain=false 2023-05-02 02:55:28,694 INFO [jenkins-hbase4:41119] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-05-02 02:55:28,696 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=8aefa099a2e821000b0b58916b550287, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:28,696 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1682996127823.8aefa099a2e821000b0b58916b550287.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1682996128696"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1682996128696"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682996128696"}]},"ts":"1682996128696"} 2023-05-02 02:55:28,698 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=133, ppid=132, state=RUNNABLE; OpenRegionProcedure 8aefa099a2e821000b0b58916b550287, server=jenkins-hbase4.apache.org,39081,1682996021579}] 2023-05-02 02:55:28,700 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=d317f7d995153dcea00ba9ce59b06174, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:28,700 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1682996127823.d317f7d995153dcea00ba9ce59b06174.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1682996128700"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1682996128700"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682996128700"}]},"ts":"1682996128700"} 2023-05-02 02:55:28,701 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=134, ppid=131, state=RUNNABLE; OpenRegionProcedure d317f7d995153dcea00ba9ce59b06174, server=jenkins-hbase4.apache.org,39081,1682996021579}] 2023-05-02 02:55:28,851 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionEmptyDaughter,71,1682996127823.8aefa099a2e821000b0b58916b550287. 2023-05-02 02:55:28,852 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 8aefa099a2e821000b0b58916b550287, NAME => 'testSplitTableRegionEmptyDaughter,71,1682996127823.8aefa099a2e821000b0b58916b550287.', STARTKEY => '71', ENDKEY => ''} 2023-05-02 02:55:28,852 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-02 02:55:28,852 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 8aefa099a2e821000b0b58916b550287 2023-05-02 02:55:28,852 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionEmptyDaughter,71,1682996127823.8aefa099a2e821000b0b58916b550287.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-02 02:55:28,852 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 8aefa099a2e821000b0b58916b550287 2023-05-02 02:55:28,852 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 8aefa099a2e821000b0b58916b550287 2023-05-02 02:55:28,853 INFO [StoreOpener-8aefa099a2e821000b0b58916b550287-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 8aefa099a2e821000b0b58916b550287 2023-05-02 02:55:28,854 DEBUG [StoreOpener-8aefa099a2e821000b0b58916b550287-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/8aefa099a2e821000b0b58916b550287/cf1 2023-05-02 02:55:28,854 DEBUG [StoreOpener-8aefa099a2e821000b0b58916b550287-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/8aefa099a2e821000b0b58916b550287/cf1 2023-05-02 02:55:28,855 INFO [StoreOpener-8aefa099a2e821000b0b58916b550287-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8aefa099a2e821000b0b58916b550287 columnFamilyName cf1 2023-05-02 02:55:28,855 INFO [StoreOpener-8aefa099a2e821000b0b58916b550287-1] regionserver.HStore(310): Store=8aefa099a2e821000b0b58916b550287/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-02 02:55:28,855 INFO [StoreOpener-8aefa099a2e821000b0b58916b550287-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 8aefa099a2e821000b0b58916b550287 2023-05-02 02:55:28,856 DEBUG [StoreOpener-8aefa099a2e821000b0b58916b550287-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/8aefa099a2e821000b0b58916b550287/cf2 2023-05-02 02:55:28,856 DEBUG [StoreOpener-8aefa099a2e821000b0b58916b550287-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/8aefa099a2e821000b0b58916b550287/cf2 2023-05-02 02:55:28,857 INFO [StoreOpener-8aefa099a2e821000b0b58916b550287-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8aefa099a2e821000b0b58916b550287 columnFamilyName cf2 2023-05-02 02:55:28,857 INFO [StoreOpener-8aefa099a2e821000b0b58916b550287-1] regionserver.HStore(310): Store=8aefa099a2e821000b0b58916b550287/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-02 02:55:28,858 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/8aefa099a2e821000b0b58916b550287 2023-05-02 02:55:28,858 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/8aefa099a2e821000b0b58916b550287 2023-05-02 02:55:28,860 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-02 02:55:28,861 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 8aefa099a2e821000b0b58916b550287 2023-05-02 02:55:28,861 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 8aefa099a2e821000b0b58916b550287; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73748454, jitterRate=0.09893760085105896}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-02 02:55:28,861 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 8aefa099a2e821000b0b58916b550287: 2023-05-02 02:55:28,862 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,71,1682996127823.8aefa099a2e821000b0b58916b550287., pid=133, masterSystemTime=1682996128849 2023-05-02 02:55:28,863 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,71,1682996127823.8aefa099a2e821000b0b58916b550287. 2023-05-02 02:55:28,863 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionEmptyDaughter,71,1682996127823.8aefa099a2e821000b0b58916b550287. 2023-05-02 02:55:28,863 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionEmptyDaughter,,1682996127823.d317f7d995153dcea00ba9ce59b06174. 2023-05-02 02:55:28,863 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => d317f7d995153dcea00ba9ce59b06174, NAME => 'testSplitTableRegionEmptyDaughter,,1682996127823.d317f7d995153dcea00ba9ce59b06174.', STARTKEY => '', ENDKEY => '71'} 2023-05-02 02:55:28,864 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=8aefa099a2e821000b0b58916b550287, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:28,864 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-02 02:55:28,864 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,71,1682996127823.8aefa099a2e821000b0b58916b550287.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1682996128864"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1682996128864"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1682996128864"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682996128864"}]},"ts":"1682996128864"} 2023-05-02 02:55:28,864 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter d317f7d995153dcea00ba9ce59b06174 2023-05-02 02:55:28,864 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionEmptyDaughter,,1682996127823.d317f7d995153dcea00ba9ce59b06174.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-02 02:55:28,864 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for d317f7d995153dcea00ba9ce59b06174 2023-05-02 02:55:28,864 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for d317f7d995153dcea00ba9ce59b06174 2023-05-02 02:55:28,865 INFO [StoreOpener-d317f7d995153dcea00ba9ce59b06174-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region d317f7d995153dcea00ba9ce59b06174 2023-05-02 02:55:28,866 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=133, resume processing ppid=132 2023-05-02 02:55:28,866 DEBUG [StoreOpener-d317f7d995153dcea00ba9ce59b06174-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf1 2023-05-02 02:55:28,866 DEBUG [StoreOpener-d317f7d995153dcea00ba9ce59b06174-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf1 2023-05-02 02:55:28,866 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=133, ppid=132, state=SUCCESS; OpenRegionProcedure 8aefa099a2e821000b0b58916b550287, server=jenkins-hbase4.apache.org,39081,1682996021579 in 168 msec 2023-05-02 02:55:28,867 INFO [StoreOpener-d317f7d995153dcea00ba9ce59b06174-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d317f7d995153dcea00ba9ce59b06174 columnFamilyName cf1 2023-05-02 02:55:28,867 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=132, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=8aefa099a2e821000b0b58916b550287, ASSIGN in 323 msec 2023-05-02 02:55:28,873 DEBUG [StoreOpener-d317f7d995153dcea00ba9ce59b06174-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf1/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-05ef4a10ee5f4a17bac25ec17d04e9bb 2023-05-02 02:55:28,875 DEBUG [StoreOpener-d317f7d995153dcea00ba9ce59b06174-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf1/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-133b1126a5fa48ca9e2e56e2e5617012 2023-05-02 02:55:28,878 DEBUG [StoreOpener-d317f7d995153dcea00ba9ce59b06174-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf1/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-25d03ac30c914e069f0bfa4f127e2574 2023-05-02 02:55:28,880 DEBUG [StoreOpener-d317f7d995153dcea00ba9ce59b06174-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf1/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-2c99538e0b1141f192d62203b1188b2a 2023-05-02 02:55:28,882 DEBUG [StoreOpener-d317f7d995153dcea00ba9ce59b06174-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf1/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-39192fe710b14e58949a1a029ab4fb87 2023-05-02 02:55:28,885 DEBUG [StoreOpener-d317f7d995153dcea00ba9ce59b06174-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf1/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-4a91258191884da1b8c056d2aa3e968b 2023-05-02 02:55:28,887 DEBUG [StoreOpener-d317f7d995153dcea00ba9ce59b06174-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf1/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-fdc47bf1839746f685697b19b502c6e1 2023-05-02 02:55:28,888 INFO [StoreOpener-d317f7d995153dcea00ba9ce59b06174-1] regionserver.HStore(310): Store=d317f7d995153dcea00ba9ce59b06174/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-02 02:55:28,888 INFO [StoreOpener-d317f7d995153dcea00ba9ce59b06174-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region d317f7d995153dcea00ba9ce59b06174 2023-05-02 02:55:28,889 DEBUG [StoreOpener-d317f7d995153dcea00ba9ce59b06174-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf2 2023-05-02 02:55:28,889 DEBUG [StoreOpener-d317f7d995153dcea00ba9ce59b06174-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf2 2023-05-02 02:55:28,889 INFO [StoreOpener-d317f7d995153dcea00ba9ce59b06174-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d317f7d995153dcea00ba9ce59b06174 columnFamilyName cf2 2023-05-02 02:55:28,894 DEBUG [StoreOpener-d317f7d995153dcea00ba9ce59b06174-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf2/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-0a3e2789ee70441192f6dbc5f13b73b4 2023-05-02 02:55:28,897 DEBUG [StoreOpener-d317f7d995153dcea00ba9ce59b06174-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf2/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-202e8eb538bb4f4aac7238dd9066a205 2023-05-02 02:55:28,899 DEBUG [StoreOpener-d317f7d995153dcea00ba9ce59b06174-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf2/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-3f6de7b2a6364f838124fe62355ae13f 2023-05-02 02:55:28,901 DEBUG [StoreOpener-d317f7d995153dcea00ba9ce59b06174-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf2/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-6cc34836bb60497cb8baefa422d13bcd 2023-05-02 02:55:28,904 DEBUG [StoreOpener-d317f7d995153dcea00ba9ce59b06174-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf2/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-7bdb429f97934894a104bf928b86330d 2023-05-02 02:55:28,906 DEBUG [StoreOpener-d317f7d995153dcea00ba9ce59b06174-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf2/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-a248d1981ba743d29600881f0ac53928 2023-05-02 02:55:28,908 DEBUG [StoreOpener-d317f7d995153dcea00ba9ce59b06174-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf2/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-dbff77c7426e41e5b02bac811d6395c6 2023-05-02 02:55:28,909 INFO [StoreOpener-d317f7d995153dcea00ba9ce59b06174-1] regionserver.HStore(310): Store=d317f7d995153dcea00ba9ce59b06174/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-02 02:55:28,909 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174 2023-05-02 02:55:28,910 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174 2023-05-02 02:55:28,911 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-02 02:55:28,912 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for d317f7d995153dcea00ba9ce59b06174 2023-05-02 02:55:28,913 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened d317f7d995153dcea00ba9ce59b06174; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71249605, jitterRate=0.06170184910297394}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-02 02:55:28,913 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for d317f7d995153dcea00ba9ce59b06174: 2023-05-02 02:55:28,914 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1682996127823.d317f7d995153dcea00ba9ce59b06174., pid=134, masterSystemTime=1682996128849 2023-05-02 02:55:28,914 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-02 02:55:28,915 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-05-02 02:55:28,915 DEBUG [RS:0;jenkins-hbase4:39081-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-02 02:55:28,915 DEBUG [RS:0;jenkins-hbase4:39081-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-02 02:55:28,915 DEBUG [RS:0;jenkins-hbase4:39081-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-05-02 02:55:28,915 DEBUG [RS:0;jenkins-hbase4:39081-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-05-02 02:55:28,915 DEBUG [RS:0;jenkins-hbase4:39081-shortCompactions-0] regionserver.HStore(1912): d317f7d995153dcea00ba9ce59b06174/cf1 is initiating minor compaction (all files) 2023-05-02 02:55:28,915 DEBUG [RS:0;jenkins-hbase4:39081-longCompactions-0] regionserver.HStore(1912): d317f7d995153dcea00ba9ce59b06174/cf2 is initiating minor compaction (all files) 2023-05-02 02:55:28,915 INFO [RS:0;jenkins-hbase4:39081-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of d317f7d995153dcea00ba9ce59b06174/cf1 in testSplitTableRegionEmptyDaughter,,1682996127823.d317f7d995153dcea00ba9ce59b06174. 2023-05-02 02:55:28,915 INFO [RS:0;jenkins-hbase4:39081-longCompactions-0] regionserver.HRegion(2259): Starting compaction of d317f7d995153dcea00ba9ce59b06174/cf2 in testSplitTableRegionEmptyDaughter,,1682996127823.d317f7d995153dcea00ba9ce59b06174. 2023-05-02 02:55:28,915 INFO [RS:0;jenkins-hbase4:39081-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf1/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-4a91258191884da1b8c056d2aa3e968b, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf1/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-05ef4a10ee5f4a17bac25ec17d04e9bb, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf1/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-133b1126a5fa48ca9e2e56e2e5617012, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf1/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-2c99538e0b1141f192d62203b1188b2a, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf1/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-fdc47bf1839746f685697b19b502c6e1, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf1/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-39192fe710b14e58949a1a029ab4fb87, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf1/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-25d03ac30c914e069f0bfa4f127e2574] into tmpdir=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/.tmp, totalSize=34.9 K 2023-05-02 02:55:28,915 INFO [RS:0;jenkins-hbase4:39081-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf2/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-0a3e2789ee70441192f6dbc5f13b73b4, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf2/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-3f6de7b2a6364f838124fe62355ae13f, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf2/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-6cc34836bb60497cb8baefa422d13bcd, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf2/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-202e8eb538bb4f4aac7238dd9066a205, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf2/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-7bdb429f97934894a104bf928b86330d, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf2/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-a248d1981ba743d29600881f0ac53928, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf2/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-dbff77c7426e41e5b02bac811d6395c6] into tmpdir=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/.tmp, totalSize=34.9 K 2023-05-02 02:55:28,916 DEBUG [RS:0;jenkins-hbase4:39081-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-4a91258191884da1b8c056d2aa3e968b, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1682996126386 2023-05-02 02:55:28,916 DEBUG [RS:0;jenkins-hbase4:39081-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-0a3e2789ee70441192f6dbc5f13b73b4, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1682996126386 2023-05-02 02:55:28,916 DEBUG [RS:0;jenkins-hbase4:39081-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-05ef4a10ee5f4a17bac25ec17d04e9bb, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1682996126552 2023-05-02 02:55:28,916 DEBUG [RS:0;jenkins-hbase4:39081-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-3f6de7b2a6364f838124fe62355ae13f, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1682996126552 2023-05-02 02:55:28,917 DEBUG [RS:0;jenkins-hbase4:39081-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-133b1126a5fa48ca9e2e56e2e5617012, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1682996126718 2023-05-02 02:55:28,917 DEBUG [RS:0;jenkins-hbase4:39081-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-6cc34836bb60497cb8baefa422d13bcd, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1682996126718 2023-05-02 02:55:28,917 DEBUG [RS:0;jenkins-hbase4:39081-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-2c99538e0b1141f192d62203b1188b2a, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1682996127297 2023-05-02 02:55:28,917 DEBUG [RS:0;jenkins-hbase4:39081-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-202e8eb538bb4f4aac7238dd9066a205, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1682996127297 2023-05-02 02:55:28,917 DEBUG [RS:0;jenkins-hbase4:39081-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-fdc47bf1839746f685697b19b502c6e1, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1682996127480 2023-05-02 02:55:28,918 DEBUG [RS:0;jenkins-hbase4:39081-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-7bdb429f97934894a104bf928b86330d, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1682996127480 2023-05-02 02:55:28,918 DEBUG [RS:0;jenkins-hbase4:39081-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-39192fe710b14e58949a1a029ab4fb87, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1682996127646 2023-05-02 02:55:28,918 DEBUG [RS:0;jenkins-hbase4:39081-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-a248d1981ba743d29600881f0ac53928, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1682996127646 2023-05-02 02:55:28,919 DEBUG [RS:0;jenkins-hbase4:39081-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-dbff77c7426e41e5b02bac811d6395c6, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1682996127816 2023-05-02 02:55:28,919 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=d317f7d995153dcea00ba9ce59b06174, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:28,919 DEBUG [RS:0;jenkins-hbase4:39081-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-25d03ac30c914e069f0bfa4f127e2574, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1682996127816 2023-05-02 02:55:28,919 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1682996127823.d317f7d995153dcea00ba9ce59b06174. 2023-05-02 02:55:28,920 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionEmptyDaughter,,1682996127823.d317f7d995153dcea00ba9ce59b06174. 2023-05-02 02:55:28,919 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1682996127823.d317f7d995153dcea00ba9ce59b06174.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1682996128919"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1682996128919"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1682996128919"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682996128919"}]},"ts":"1682996128919"} 2023-05-02 02:55:28,923 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=134, resume processing ppid=131 2023-05-02 02:55:28,923 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=134, ppid=131, state=SUCCESS; OpenRegionProcedure d317f7d995153dcea00ba9ce59b06174, server=jenkins-hbase4.apache.org,39081,1682996021579 in 220 msec 2023-05-02 02:55:28,925 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=131, resume processing ppid=128 2023-05-02 02:55:28,925 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=131, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=d317f7d995153dcea00ba9ce59b06174, ASSIGN in 380 msec 2023-05-02 02:55:28,926 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=128, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=d41bae7ce64ef0743e56e5fbc14a3808, daughterA=d317f7d995153dcea00ba9ce59b06174, daughterB=8aefa099a2e821000b0b58916b550287 in 1.1020 sec 2023-05-02 02:55:28,952 INFO [RS:0;jenkins-hbase4:39081-shortCompactions-0] throttle.PressureAwareThroughputController(145): d317f7d995153dcea00ba9ce59b06174#cf1#compaction#126 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-02 02:55:28,964 INFO [RS:0;jenkins-hbase4:39081-longCompactions-0] throttle.PressureAwareThroughputController(145): d317f7d995153dcea00ba9ce59b06174#cf2#compaction#127 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-02 02:55:28,972 DEBUG [RS:0;jenkins-hbase4:39081-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/.tmp/cf1/f964bd19a7b74d7a8c8dd3b1f3443863 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf1/f964bd19a7b74d7a8c8dd3b1f3443863 2023-05-02 02:55:28,976 INFO [RS:0;jenkins-hbase4:39081-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in d317f7d995153dcea00ba9ce59b06174/cf1 of d317f7d995153dcea00ba9ce59b06174 into f964bd19a7b74d7a8c8dd3b1f3443863(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-02 02:55:28,976 DEBUG [RS:0;jenkins-hbase4:39081-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for d317f7d995153dcea00ba9ce59b06174: 2023-05-02 02:55:28,976 INFO [RS:0;jenkins-hbase4:39081-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1682996127823.d317f7d995153dcea00ba9ce59b06174., storeName=d317f7d995153dcea00ba9ce59b06174/cf1, priority=9, startTime=1682996128914; duration=0sec 2023-05-02 02:55:28,977 DEBUG [RS:0;jenkins-hbase4:39081-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-02 02:55:29,075 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39081] ipc.CallRunner(144): callId: 987 service: ClientService methodName: Scan size: 157 connection: 172.31.14.131:52194 deadline: 1682996189075, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionEmptyDaughter,,1682996125882.d41bae7ce64ef0743e56e5fbc14a3808. is not online on jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:29,194 DEBUG [Listener at localhost/37119] client.ClientScanner(240): Advancing internal scanner to startKey at '71', inclusive 2023-05-02 02:55:29,197 WARN [Listener at localhost/37119] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-02 02:55:29,198 INFO [Listener at localhost/37119] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionEmptyDaughter 2023-05-02 02:55:29,199 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testSplitTableRegionEmptyDaughter 2023-05-02 02:55:29,199 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure2.ProcedureExecutor(1029): Stored pid=135, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionEmptyDaughter 2023-05-02 02:55:29,201 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-05-02 02:55:29,201 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682996129201"}]},"ts":"1682996129201"} 2023-05-02 02:55:29,202 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLING in hbase:meta 2023-05-02 02:55:29,204 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionEmptyDaughter to state=DISABLING 2023-05-02 02:55:29,204 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=d317f7d995153dcea00ba9ce59b06174, UNASSIGN}, {pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=8aefa099a2e821000b0b58916b550287, UNASSIGN}] 2023-05-02 02:55:29,205 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=8aefa099a2e821000b0b58916b550287, UNASSIGN 2023-05-02 02:55:29,205 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=8aefa099a2e821000b0b58916b550287, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:29,205 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1682996127823.8aefa099a2e821000b0b58916b550287.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1682996129205"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1682996129205"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682996129205"}]},"ts":"1682996129205"} 2023-05-02 02:55:29,206 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=138, ppid=137, state=RUNNABLE; CloseRegionProcedure 8aefa099a2e821000b0b58916b550287, server=jenkins-hbase4.apache.org,39081,1682996021579}] 2023-05-02 02:55:29,208 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=d317f7d995153dcea00ba9ce59b06174, UNASSIGN 2023-05-02 02:55:29,208 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=d317f7d995153dcea00ba9ce59b06174, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:29,208 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1682996127823.d317f7d995153dcea00ba9ce59b06174.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1682996129208"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1682996129208"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682996129208"}]},"ts":"1682996129208"} 2023-05-02 02:55:29,209 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=139, ppid=136, state=RUNNABLE; CloseRegionProcedure d317f7d995153dcea00ba9ce59b06174, server=jenkins-hbase4.apache.org,39081,1682996021579}] 2023-05-02 02:55:29,212 ERROR [BP-376276558-172.31.14.131-1682996016523 heartbeating to localhost/127.0.0.1:38649] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-376276558-172.31.14.131-1682996016523 (Datanode Uuid 12e74f69-84cc-44e8-ac7e-14ebfb93f6fc) service to localhost/127.0.0.1:38649 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-05-02 02:55:29,302 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-05-02 02:55:29,357 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 8aefa099a2e821000b0b58916b550287 2023-05-02 02:55:29,358 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 8aefa099a2e821000b0b58916b550287, disabling compactions & flushes 2023-05-02 02:55:29,358 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionEmptyDaughter,71,1682996127823.8aefa099a2e821000b0b58916b550287. 2023-05-02 02:55:29,358 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,71,1682996127823.8aefa099a2e821000b0b58916b550287. 2023-05-02 02:55:29,358 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionEmptyDaughter,71,1682996127823.8aefa099a2e821000b0b58916b550287. after waiting 0 ms 2023-05-02 02:55:29,358 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionEmptyDaughter,71,1682996127823.8aefa099a2e821000b0b58916b550287. 2023-05-02 02:55:29,361 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/8aefa099a2e821000b0b58916b550287/recovered.edits/88.seqid, newMaxSeqId=88, maxSeqId=85 2023-05-02 02:55:29,362 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-02 02:55:29,362 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionEmptyDaughter,71,1682996127823.8aefa099a2e821000b0b58916b550287. 2023-05-02 02:55:29,362 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 8aefa099a2e821000b0b58916b550287: 2023-05-02 02:55:29,363 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 8aefa099a2e821000b0b58916b550287 2023-05-02 02:55:29,363 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close d317f7d995153dcea00ba9ce59b06174 2023-05-02 02:55:29,363 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing d317f7d995153dcea00ba9ce59b06174, disabling compactions & flushes 2023-05-02 02:55:29,363 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1858): waiting for 1 compactions to complete for region testSplitTableRegionEmptyDaughter,,1682996127823.d317f7d995153dcea00ba9ce59b06174. 2023-05-02 02:55:29,364 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=8aefa099a2e821000b0b58916b550287, regionState=CLOSED 2023-05-02 02:55:29,364 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,71,1682996127823.8aefa099a2e821000b0b58916b550287.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1682996129363"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682996129363"}]},"ts":"1682996129363"} 2023-05-02 02:55:29,366 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=138, resume processing ppid=137 2023-05-02 02:55:29,366 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=138, ppid=137, state=SUCCESS; CloseRegionProcedure 8aefa099a2e821000b0b58916b550287, server=jenkins-hbase4.apache.org,39081,1682996021579 in 159 msec 2023-05-02 02:55:29,367 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=137, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=8aefa099a2e821000b0b58916b550287, UNASSIGN in 162 msec 2023-05-02 02:55:29,379 DEBUG [RS:0;jenkins-hbase4:39081-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/.tmp/cf2/d90983c682e343a7ba791f5013286771 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf2/d90983c682e343a7ba791f5013286771 2023-05-02 02:55:29,383 INFO [RS:0;jenkins-hbase4:39081-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in d317f7d995153dcea00ba9ce59b06174/cf2 of d317f7d995153dcea00ba9ce59b06174 into d90983c682e343a7ba791f5013286771(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-02 02:55:29,383 DEBUG [RS:0;jenkins-hbase4:39081-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for d317f7d995153dcea00ba9ce59b06174: 2023-05-02 02:55:29,383 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionEmptyDaughter,,1682996127823.d317f7d995153dcea00ba9ce59b06174. 2023-05-02 02:55:29,383 INFO [RS:0;jenkins-hbase4:39081-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1682996127823.d317f7d995153dcea00ba9ce59b06174., storeName=d317f7d995153dcea00ba9ce59b06174/cf2, priority=9, startTime=1682996128915; duration=0sec 2023-05-02 02:55:29,383 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1682996127823.d317f7d995153dcea00ba9ce59b06174. 2023-05-02 02:55:29,383 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionEmptyDaughter,,1682996127823.d317f7d995153dcea00ba9ce59b06174. after waiting 0 ms 2023-05-02 02:55:29,383 DEBUG [RS:0;jenkins-hbase4:39081-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-02 02:55:29,383 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionEmptyDaughter,,1682996127823.d317f7d995153dcea00ba9ce59b06174. 2023-05-02 02:55:29,384 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1682996127823.d317f7d995153dcea00ba9ce59b06174.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf1/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-4a91258191884da1b8c056d2aa3e968b, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf1/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-05ef4a10ee5f4a17bac25ec17d04e9bb, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf1/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-133b1126a5fa48ca9e2e56e2e5617012, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf1/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-2c99538e0b1141f192d62203b1188b2a, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf1/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-fdc47bf1839746f685697b19b502c6e1, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf1/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-39192fe710b14e58949a1a029ab4fb87, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf1/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-25d03ac30c914e069f0bfa4f127e2574] to archive 2023-05-02 02:55:29,385 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1682996127823.d317f7d995153dcea00ba9ce59b06174.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-02 02:55:29,386 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1682996127823.d317f7d995153dcea00ba9ce59b06174.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf1/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-4a91258191884da1b8c056d2aa3e968b to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf1/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-4a91258191884da1b8c056d2aa3e968b 2023-05-02 02:55:29,387 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1682996127823.d317f7d995153dcea00ba9ce59b06174.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf1/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-05ef4a10ee5f4a17bac25ec17d04e9bb to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf1/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-05ef4a10ee5f4a17bac25ec17d04e9bb 2023-05-02 02:55:29,389 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1682996127823.d317f7d995153dcea00ba9ce59b06174.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf1/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-133b1126a5fa48ca9e2e56e2e5617012 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf1/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-133b1126a5fa48ca9e2e56e2e5617012 2023-05-02 02:55:29,390 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1682996127823.d317f7d995153dcea00ba9ce59b06174.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf1/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-2c99538e0b1141f192d62203b1188b2a to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf1/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-2c99538e0b1141f192d62203b1188b2a 2023-05-02 02:55:29,391 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1682996127823.d317f7d995153dcea00ba9ce59b06174.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf1/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-fdc47bf1839746f685697b19b502c6e1 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf1/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-fdc47bf1839746f685697b19b502c6e1 2023-05-02 02:55:29,392 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1682996127823.d317f7d995153dcea00ba9ce59b06174.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf1/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-39192fe710b14e58949a1a029ab4fb87 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf1/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-39192fe710b14e58949a1a029ab4fb87 2023-05-02 02:55:29,393 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1682996127823.d317f7d995153dcea00ba9ce59b06174.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf1/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-25d03ac30c914e069f0bfa4f127e2574 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf1/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-25d03ac30c914e069f0bfa4f127e2574 2023-05-02 02:55:29,394 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1682996127823.d317f7d995153dcea00ba9ce59b06174.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf2/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-0a3e2789ee70441192f6dbc5f13b73b4, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf2/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-3f6de7b2a6364f838124fe62355ae13f, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf2/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-6cc34836bb60497cb8baefa422d13bcd, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf2/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-202e8eb538bb4f4aac7238dd9066a205, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf2/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-7bdb429f97934894a104bf928b86330d, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf2/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-a248d1981ba743d29600881f0ac53928, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf2/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-dbff77c7426e41e5b02bac811d6395c6] to archive 2023-05-02 02:55:29,394 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1682996127823.d317f7d995153dcea00ba9ce59b06174.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-02 02:55:29,396 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1682996127823.d317f7d995153dcea00ba9ce59b06174.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf2/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-0a3e2789ee70441192f6dbc5f13b73b4 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf2/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-0a3e2789ee70441192f6dbc5f13b73b4 2023-05-02 02:55:29,397 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1682996127823.d317f7d995153dcea00ba9ce59b06174.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf2/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-3f6de7b2a6364f838124fe62355ae13f to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf2/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-3f6de7b2a6364f838124fe62355ae13f 2023-05-02 02:55:29,398 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1682996127823.d317f7d995153dcea00ba9ce59b06174.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf2/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-6cc34836bb60497cb8baefa422d13bcd to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf2/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-6cc34836bb60497cb8baefa422d13bcd 2023-05-02 02:55:29,399 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1682996127823.d317f7d995153dcea00ba9ce59b06174.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf2/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-202e8eb538bb4f4aac7238dd9066a205 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf2/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-202e8eb538bb4f4aac7238dd9066a205 2023-05-02 02:55:29,400 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1682996127823.d317f7d995153dcea00ba9ce59b06174.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf2/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-7bdb429f97934894a104bf928b86330d to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf2/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-7bdb429f97934894a104bf928b86330d 2023-05-02 02:55:29,401 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1682996127823.d317f7d995153dcea00ba9ce59b06174.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf2/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-a248d1981ba743d29600881f0ac53928 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf2/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-a248d1981ba743d29600881f0ac53928 2023-05-02 02:55:29,402 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1682996127823.d317f7d995153dcea00ba9ce59b06174.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf2/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-dbff77c7426e41e5b02bac811d6395c6 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf2/testSplitTableRegionEmptyDaughter=d41bae7ce64ef0743e56e5fbc14a3808-dbff77c7426e41e5b02bac811d6395c6 2023-05-02 02:55:29,405 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-05-02 02:55:29,406 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-02 02:55:29,406 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionEmptyDaughter,,1682996127823.d317f7d995153dcea00ba9ce59b06174. 2023-05-02 02:55:29,406 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for d317f7d995153dcea00ba9ce59b06174: 2023-05-02 02:55:29,407 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed d317f7d995153dcea00ba9ce59b06174 2023-05-02 02:55:29,407 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=d317f7d995153dcea00ba9ce59b06174, regionState=CLOSED 2023-05-02 02:55:29,407 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1682996127823.d317f7d995153dcea00ba9ce59b06174.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1682996129407"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682996129407"}]},"ts":"1682996129407"} 2023-05-02 02:55:29,409 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=139, resume processing ppid=136 2023-05-02 02:55:29,409 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=139, ppid=136, state=SUCCESS; CloseRegionProcedure d317f7d995153dcea00ba9ce59b06174, server=jenkins-hbase4.apache.org,39081,1682996021579 in 199 msec 2023-05-02 02:55:29,410 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=136, resume processing ppid=135 2023-05-02 02:55:29,411 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=136, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=d317f7d995153dcea00ba9ce59b06174, UNASSIGN in 205 msec 2023-05-02 02:55:29,411 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682996129411"}]},"ts":"1682996129411"} 2023-05-02 02:55:29,412 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLED in hbase:meta 2023-05-02 02:55:29,414 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionEmptyDaughter to state=DISABLED 2023-05-02 02:55:29,415 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=135, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionEmptyDaughter in 215 msec 2023-05-02 02:55:29,503 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-05-02 02:55:29,504 INFO [Listener at localhost/37119] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 135 completed 2023-05-02 02:55:29,504 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testSplitTableRegionEmptyDaughter 2023-05-02 02:55:29,505 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure2.ProcedureExecutor(1029): Stored pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-05-02 02:55:29,507 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-05-02 02:55:29,507 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1227): Checking to see if procedure is done pid=140 2023-05-02 02:55:29,507 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=140, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-05-02 02:55:29,511 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808 2023-05-02 02:55:29,511 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionEmptyDaughter/8aefa099a2e821000b0b58916b550287 2023-05-02 02:55:29,511 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174 2023-05-02 02:55:29,513 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf1, FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf2, FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/recovered.edits] 2023-05-02 02:55:29,513 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionEmptyDaughter/8aefa099a2e821000b0b58916b550287/cf1, FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionEmptyDaughter/8aefa099a2e821000b0b58916b550287/cf2, FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionEmptyDaughter/8aefa099a2e821000b0b58916b550287/recovered.edits] 2023-05-02 02:55:29,513 DEBUG [HFileArchiver-10] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf1, FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf2, FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/recovered.edits] 2023-05-02 02:55:29,516 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf1/05ef4a10ee5f4a17bac25ec17d04e9bb to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf1/05ef4a10ee5f4a17bac25ec17d04e9bb 2023-05-02 02:55:29,516 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf1/f964bd19a7b74d7a8c8dd3b1f3443863 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf1/f964bd19a7b74d7a8c8dd3b1f3443863 2023-05-02 02:55:29,517 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf1/133b1126a5fa48ca9e2e56e2e5617012 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf1/133b1126a5fa48ca9e2e56e2e5617012 2023-05-02 02:55:29,518 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionEmptyDaughter/8aefa099a2e821000b0b58916b550287/recovered.edits/88.seqid to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionEmptyDaughter/8aefa099a2e821000b0b58916b550287/recovered.edits/88.seqid 2023-05-02 02:55:29,518 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf2/d90983c682e343a7ba791f5013286771 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/cf2/d90983c682e343a7ba791f5013286771 2023-05-02 02:55:29,519 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionEmptyDaughter/8aefa099a2e821000b0b58916b550287 2023-05-02 02:55:29,519 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf1/25d03ac30c914e069f0bfa4f127e2574 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf1/25d03ac30c914e069f0bfa4f127e2574 2023-05-02 02:55:29,520 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf1/2c99538e0b1141f192d62203b1188b2a to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf1/2c99538e0b1141f192d62203b1188b2a 2023-05-02 02:55:29,521 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/recovered.edits/90.seqid to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174/recovered.edits/90.seqid 2023-05-02 02:55:29,521 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf1/39192fe710b14e58949a1a029ab4fb87 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf1/39192fe710b14e58949a1a029ab4fb87 2023-05-02 02:55:29,522 DEBUG [HFileArchiver-10] backup.HFileArchiver(596): Deleted hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionEmptyDaughter/d317f7d995153dcea00ba9ce59b06174 2023-05-02 02:55:29,522 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf1/4a91258191884da1b8c056d2aa3e968b to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf1/4a91258191884da1b8c056d2aa3e968b 2023-05-02 02:55:29,523 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf1/fdc47bf1839746f685697b19b502c6e1 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf1/fdc47bf1839746f685697b19b502c6e1 2023-05-02 02:55:29,525 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf2/0a3e2789ee70441192f6dbc5f13b73b4 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf2/0a3e2789ee70441192f6dbc5f13b73b4 2023-05-02 02:55:29,526 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf2/202e8eb538bb4f4aac7238dd9066a205 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf2/202e8eb538bb4f4aac7238dd9066a205 2023-05-02 02:55:29,527 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf2/3f6de7b2a6364f838124fe62355ae13f to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf2/3f6de7b2a6364f838124fe62355ae13f 2023-05-02 02:55:29,528 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf2/6cc34836bb60497cb8baefa422d13bcd to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf2/6cc34836bb60497cb8baefa422d13bcd 2023-05-02 02:55:29,529 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf2/7bdb429f97934894a104bf928b86330d to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf2/7bdb429f97934894a104bf928b86330d 2023-05-02 02:55:29,530 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf2/a248d1981ba743d29600881f0ac53928 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf2/a248d1981ba743d29600881f0ac53928 2023-05-02 02:55:29,531 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf2/dbff77c7426e41e5b02bac811d6395c6 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/cf2/dbff77c7426e41e5b02bac811d6395c6 2023-05-02 02:55:29,534 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/recovered.edits/85.seqid to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808/recovered.edits/85.seqid 2023-05-02 02:55:29,534 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegionEmptyDaughter/d41bae7ce64ef0743e56e5fbc14a3808 2023-05-02 02:55:29,534 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-05-02 02:55:29,536 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=140, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-05-02 02:55:29,538 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionEmptyDaughter from hbase:meta 2023-05-02 02:55:29,539 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionEmptyDaughter' descriptor. 2023-05-02 02:55:29,540 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=140, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-05-02 02:55:29,540 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionEmptyDaughter' from region states. 2023-05-02 02:55:29,541 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1682996125882.d41bae7ce64ef0743e56e5fbc14a3808.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1682996129540"}]},"ts":"9223372036854775807"} 2023-05-02 02:55:29,541 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1682996127823.d317f7d995153dcea00ba9ce59b06174.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1682996129540"}]},"ts":"9223372036854775807"} 2023-05-02 02:55:29,541 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,71,1682996127823.8aefa099a2e821000b0b58916b550287.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1682996129540"}]},"ts":"9223372036854775807"} 2023-05-02 02:55:29,542 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-05-02 02:55:29,542 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => d41bae7ce64ef0743e56e5fbc14a3808, NAME => 'testSplitTableRegionEmptyDaughter,,1682996125882.d41bae7ce64ef0743e56e5fbc14a3808.', STARTKEY => '', ENDKEY => ''}, {ENCODED => d317f7d995153dcea00ba9ce59b06174, NAME => 'testSplitTableRegionEmptyDaughter,,1682996127823.d317f7d995153dcea00ba9ce59b06174.', STARTKEY => '', ENDKEY => '71'}, {ENCODED => 8aefa099a2e821000b0b58916b550287, NAME => 'testSplitTableRegionEmptyDaughter,71,1682996127823.8aefa099a2e821000b0b58916b550287.', STARTKEY => '71', ENDKEY => ''}] 2023-05-02 02:55:29,542 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionEmptyDaughter' as deleted. 2023-05-02 02:55:29,542 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1682996129542"}]},"ts":"9223372036854775807"} 2023-05-02 02:55:29,544 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionEmptyDaughter state from META 2023-05-02 02:55:29,545 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=140, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-05-02 02:55:29,546 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=140, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter in 41 msec 2023-05-02 02:55:29,608 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1227): Checking to see if procedure is done pid=140 2023-05-02 02:55:29,609 INFO [Listener at localhost/37119] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 140 completed 2023-05-02 02:55:29,622 INFO [Listener at localhost/37119] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=375 (was 376), OpenFileDescriptor=838 (was 838), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=159 (was 147) - SystemLoadAverage LEAK? -, ProcessCount=170 (was 170), AvailableMemoryMB=13745 (was 13749) 2023-05-02 02:55:29,635 INFO [Listener at localhost/37119] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=375, OpenFileDescriptor=838, MaxFileDescriptor=60000, SystemLoadAverage=159, ProcessCount=170, AvailableMemoryMB=13745 2023-05-02 02:55:29,635 WARN [Listener at localhost/37119] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-02 02:55:29,637 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-05-02 02:55:29,637 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-05-02 02:55:29,639 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(1029): Stored pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegion 2023-05-02 02:55:29,640 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_PRE_OPERATION 2023-05-02 02:55:29,641 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-05-02 02:55:29,642 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:29,642 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863 empty. 2023-05-02 02:55:29,642 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:29,643 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-05-02 02:55:30,052 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegion/.tabledesc/.tableinfo.0000000001 2023-05-02 02:55:30,053 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(7675): creating {ENCODED => 4b5519600ec1165492215f7e7e85c863, NAME => 'testSplitTableRegion,,1682996129638.4b5519600ec1165492215f7e7e85c863.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegion', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp 2023-05-02 02:55:30,060 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegion,,1682996129638.4b5519600ec1165492215f7e7e85c863.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-02 02:55:30,060 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1604): Closing 4b5519600ec1165492215f7e7e85c863, disabling compactions & flushes 2023-05-02 02:55:30,060 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegion,,1682996129638.4b5519600ec1165492215f7e7e85c863. 2023-05-02 02:55:30,060 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegion,,1682996129638.4b5519600ec1165492215f7e7e85c863. 2023-05-02 02:55:30,060 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegion,,1682996129638.4b5519600ec1165492215f7e7e85c863. after waiting 0 ms 2023-05-02 02:55:30,060 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegion,,1682996129638.4b5519600ec1165492215f7e7e85c863. 2023-05-02 02:55:30,060 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegion,,1682996129638.4b5519600ec1165492215f7e7e85c863. 2023-05-02 02:55:30,061 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1558): Region close journal for 4b5519600ec1165492215f7e7e85c863: 2023-05-02 02:55:30,062 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ADD_TO_META 2023-05-02 02:55:30,063 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegion,,1682996129638.4b5519600ec1165492215f7e7e85c863.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1682996130063"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682996130063"}]},"ts":"1682996130063"} 2023-05-02 02:55:30,064 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-05-02 02:55:30,065 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-05-02 02:55:30,065 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682996130065"}]},"ts":"1682996130065"} 2023-05-02 02:55:30,065 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLING in hbase:meta 2023-05-02 02:55:30,069 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-05-02 02:55:30,070 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-05-02 02:55:30,070 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-05-02 02:55:30,070 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-05-02 02:55:30,070 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-05-02 02:55:30,070 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=4b5519600ec1165492215f7e7e85c863, ASSIGN}] 2023-05-02 02:55:30,071 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=4b5519600ec1165492215f7e7e85c863, ASSIGN 2023-05-02 02:55:30,072 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=4b5519600ec1165492215f7e7e85c863, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42019,1682996021676; forceNewPlan=false, retain=false 2023-05-02 02:55:30,222 INFO [jenkins-hbase4:41119] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-05-02 02:55:30,223 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=4b5519600ec1165492215f7e7e85c863, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:30,223 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1682996129638.4b5519600ec1165492215f7e7e85c863.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1682996130223"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1682996130223"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682996130223"}]},"ts":"1682996130223"} 2023-05-02 02:55:30,225 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=143, ppid=142, state=RUNNABLE; OpenRegionProcedure 4b5519600ec1165492215f7e7e85c863, server=jenkins-hbase4.apache.org,42019,1682996021676}] 2023-05-02 02:55:30,378 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegion,,1682996129638.4b5519600ec1165492215f7e7e85c863. 2023-05-02 02:55:30,378 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 4b5519600ec1165492215f7e7e85c863, NAME => 'testSplitTableRegion,,1682996129638.4b5519600ec1165492215f7e7e85c863.', STARTKEY => '', ENDKEY => ''} 2023-05-02 02:55:30,379 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-02 02:55:30,379 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:30,379 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegion,,1682996129638.4b5519600ec1165492215f7e7e85c863.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-02 02:55:30,379 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:30,379 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:30,380 INFO [StoreOpener-4b5519600ec1165492215f7e7e85c863-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:30,381 DEBUG [StoreOpener-4b5519600ec1165492215f7e7e85c863-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1 2023-05-02 02:55:30,382 DEBUG [StoreOpener-4b5519600ec1165492215f7e7e85c863-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1 2023-05-02 02:55:30,382 INFO [StoreOpener-4b5519600ec1165492215f7e7e85c863-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4b5519600ec1165492215f7e7e85c863 columnFamilyName cf1 2023-05-02 02:55:30,382 INFO [StoreOpener-4b5519600ec1165492215f7e7e85c863-1] regionserver.HStore(310): Store=4b5519600ec1165492215f7e7e85c863/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-02 02:55:30,382 INFO [StoreOpener-4b5519600ec1165492215f7e7e85c863-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:30,383 DEBUG [StoreOpener-4b5519600ec1165492215f7e7e85c863-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2 2023-05-02 02:55:30,384 DEBUG [StoreOpener-4b5519600ec1165492215f7e7e85c863-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2 2023-05-02 02:55:30,384 INFO [StoreOpener-4b5519600ec1165492215f7e7e85c863-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4b5519600ec1165492215f7e7e85c863 columnFamilyName cf2 2023-05-02 02:55:30,384 INFO [StoreOpener-4b5519600ec1165492215f7e7e85c863-1] regionserver.HStore(310): Store=4b5519600ec1165492215f7e7e85c863/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-02 02:55:30,385 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:30,385 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:30,386 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-02 02:55:30,387 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:30,388 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-05-02 02:55:30,389 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 4b5519600ec1165492215f7e7e85c863; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70297956, jitterRate=0.04752117395401001}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-02 02:55:30,389 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 4b5519600ec1165492215f7e7e85c863: 2023-05-02 02:55:30,389 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,,1682996129638.4b5519600ec1165492215f7e7e85c863., pid=143, masterSystemTime=1682996130376 2023-05-02 02:55:30,391 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,,1682996129638.4b5519600ec1165492215f7e7e85c863. 2023-05-02 02:55:30,391 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegion,,1682996129638.4b5519600ec1165492215f7e7e85c863. 2023-05-02 02:55:30,391 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=4b5519600ec1165492215f7e7e85c863, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:30,391 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1682996129638.4b5519600ec1165492215f7e7e85c863.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1682996130391"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1682996130391"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1682996130391"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682996130391"}]},"ts":"1682996130391"} 2023-05-02 02:55:30,394 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=143, resume processing ppid=142 2023-05-02 02:55:30,394 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=143, ppid=142, state=SUCCESS; OpenRegionProcedure 4b5519600ec1165492215f7e7e85c863, server=jenkins-hbase4.apache.org,42019,1682996021676 in 167 msec 2023-05-02 02:55:30,395 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=142, resume processing ppid=141 2023-05-02 02:55:30,395 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=142, ppid=141, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=4b5519600ec1165492215f7e7e85c863, ASSIGN in 324 msec 2023-05-02 02:55:30,395 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-05-02 02:55:30,395 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682996130395"}]},"ts":"1682996130395"} 2023-05-02 02:55:30,396 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLED in hbase:meta 2023-05-02 02:55:30,398 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_POST_OPERATION 2023-05-02 02:55:30,400 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=141, state=SUCCESS; CreateTableProcedure table=testSplitTableRegion in 760 msec 2023-05-02 02:55:30,645 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-02 02:55:30,647 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-05-02 02:55:30,647 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-05-02 02:55:30,647 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:30,647 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-05-02 02:55:30,647 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-05-02 02:55:30,647 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:30,647 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-02 02:55:30,648 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:30,648 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:30,648 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:30,648 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:30,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:30,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:30,648 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:30,648 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:30,649 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:30,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:30,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-02 02:55:30,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-02 02:55:30,649 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:30,649 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-02 02:55:30,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-02 02:55:30,649 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:30,649 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:30,649 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:30,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:30,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:30,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-02 02:55:30,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-02 02:55:30,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:30,650 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-02 02:55:30,650 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-02 02:55:30,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-02 02:55:30,650 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-02 02:55:30,650 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-02 02:55:30,650 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-02 02:55:30,650 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-02 02:55:30,650 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-02 02:55:30,650 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-02 02:55:30,650 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:30,650 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-02 02:55:30,650 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-02 02:55:30,650 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-02 02:55:30,650 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-02 02:55:30,650 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:30,650 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:30,650 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-02 02:55:30,650 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-02 02:55:30,651 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-02 02:55:30,651 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38925,1682996021633' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-02 02:55:30,651 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-02 02:55:30,651 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-02 02:55:30,651 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1682996129638.4b5519600ec1165492215f7e7e85c863. 2023-05-02 02:55:30,651 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1682996129638.4b5519600ec1165492215f7e7e85c863. started... 2023-05-02 02:55:30,652 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 4b5519600ec1165492215f7e7e85c863 2/2 column families, dataSize=120 B heapSize=928 B 2023-05-02 02:55:30,652 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:30,654 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:30,654 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:30,654 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:30,655 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:30,655 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:31,063 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/.tmp/cf1/40c93f88f8df4f8e85dfd0fe36da6ea3 2023-05-02 02:55:31,074 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/.tmp/cf2/b4e5a86ddabc457190064c61898d51ca 2023-05-02 02:55:31,077 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/.tmp/cf1/40c93f88f8df4f8e85dfd0fe36da6ea3 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/40c93f88f8df4f8e85dfd0fe36da6ea3 2023-05-02 02:55:31,080 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/40c93f88f8df4f8e85dfd0fe36da6ea3, entries=2, sequenceid=6, filesize=4.8 K 2023-05-02 02:55:31,080 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/.tmp/cf2/b4e5a86ddabc457190064c61898d51ca as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/b4e5a86ddabc457190064c61898d51ca 2023-05-02 02:55:31,083 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/b4e5a86ddabc457190064c61898d51ca, entries=2, sequenceid=6, filesize=4.8 K 2023-05-02 02:55:31,084 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 4b5519600ec1165492215f7e7e85c863 in 432ms, sequenceid=6, compaction requested=false 2023-05-02 02:55:31,084 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegion' 2023-05-02 02:55:31,084 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 4b5519600ec1165492215f7e7e85c863: 2023-05-02 02:55:31,085 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1682996129638.4b5519600ec1165492215f7e7e85c863. 2023-05-02 02:55:31,085 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-02 02:55:31,085 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-02 02:55:31,085 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,085 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-02 02:55:31,085 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42019,1682996021676' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-02 02:55:31,086 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,086 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:31,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:31,087 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:31,087 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:31,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:31,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-02 02:55:31,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:31,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:31,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:31,088 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42019,1682996021676' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-05-02 02:55:31,088 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7598cc14[Count = 0] remaining members to acquire global barrier 2023-05-02 02:55:31,088 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-05-02 02:55:31,088 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,089 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,089 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,089 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,089 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,089 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,090 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-02 02:55:31,089 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,090 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-02 02:55:31,090 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-02 02:55:31,089 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,090 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-02 02:55:31,090 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,39081,1682996021579' in zk 2023-05-02 02:55:31,090 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-02 02:55:31,090 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-02 02:55:31,090 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,42019,1682996021676' in zk 2023-05-02 02:55:31,090 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,38925,1682996021633' in zk 2023-05-02 02:55:31,090 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-02 02:55:31,091 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-02 02:55:31,091 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:31,091 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-02 02:55:31,095 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-02 02:55:31,095 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,095 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:31,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,095 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-02 02:55:31,096 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:31,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:31,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:31,095 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-02 02:55:31,096 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-02 02:55:31,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:31,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:31,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-02 02:55:31,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:31,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:31,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:31,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-02 02:55:31,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:31,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:31,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,42019,1682996021676': 2023-05-02 02:55:31,098 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42019,1682996021676' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-05-02 02:55:31,098 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-05-02 02:55:31,099 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-02 02:55:31,099 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-02 02:55:31,099 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-05-02 02:55:31,099 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-02 02:55:31,100 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,100 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,100 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:31,100 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:31,100 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,100 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,100 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,100 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:31,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:31,100 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,100 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:31,100 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:31,101 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:31,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:31,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:31,100 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:31,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:31,101 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,101 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:31,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:31,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-02 02:55:31,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,101 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:31,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:31,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-02 02:55:31,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:31,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:31,103 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:31,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:31,103 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-02 02:55:31,103 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:31,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:31,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:31,105 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:31,105 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:31,105 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:31,105 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:31,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:31,106 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:31,105 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:31,106 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-05-02 02:55:31,106 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:31,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:31,105 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:31,105 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:31,107 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:31,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:31,105 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:31,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:31,107 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-02 02:55:31,106 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-02 02:55:31,106 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:31,107 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,107 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:31,107 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:31,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:31,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:31,107 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-05-02 02:55:31,107 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:31,107 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-02 02:55:31,107 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-02 02:55:31,107 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:31,107 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,107 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,107 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:31,107 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,107 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,207 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-02 02:55:31,208 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-02 02:55:31,218 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-02 02:55:31,219 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-05-02 02:55:31,220 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-05-02 02:55:31,220 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-05-02 02:55:31,220 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:31,220 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-05-02 02:55:31,220 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-05-02 02:55:31,220 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,220 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-02 02:55:31,222 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:31,222 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:31,222 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:31,222 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:31,222 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:31,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:31,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:31,222 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,222 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:31,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:31,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-02 02:55:31,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-02 02:55:31,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-02 02:55:31,223 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,223 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-02 02:55:31,223 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,223 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,223 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:31,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:31,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-02 02:55:31,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-02 02:55:31,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:31,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-02 02:55:31,223 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-02 02:55:31,223 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-02 02:55:31,223 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-02 02:55:31,224 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-05-02 02:55:31,224 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-02 02:55:31,224 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-02 02:55:31,224 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:31,224 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-05-02 02:55:31,224 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-02 02:55:31,224 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-02 02:55:31,224 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-02 02:55:31,224 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:31,224 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-02 02:55:31,224 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-02 02:55:31,224 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1682996129638.4b5519600ec1165492215f7e7e85c863. 2023-05-02 02:55:31,224 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-05-02 02:55:31,224 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1682996129638.4b5519600ec1165492215f7e7e85c863. started... 2023-05-02 02:55:31,224 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-02 02:55:31,224 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-02 02:55:31,224 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-02 02:55:31,224 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 4b5519600ec1165492215f7e7e85c863 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-02 02:55:31,224 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-02 02:55:31,224 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-02 02:55:31,224 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:31,224 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-02 02:55:31,225 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-02 02:55:31,225 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38925,1682996021633' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-02 02:55:31,226 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,226 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,226 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:31,226 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,227 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,227 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:31,233 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/.tmp/cf1/a15ebfd300ef4d08a9526159dd2f49ab 2023-05-02 02:55:31,247 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/.tmp/cf2/5e3792546819434d8bea5f8087d140de 2023-05-02 02:55:31,251 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/.tmp/cf1/a15ebfd300ef4d08a9526159dd2f49ab as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/a15ebfd300ef4d08a9526159dd2f49ab 2023-05-02 02:55:31,254 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/a15ebfd300ef4d08a9526159dd2f49ab, entries=10, sequenceid=19, filesize=5.0 K 2023-05-02 02:55:31,255 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/.tmp/cf2/5e3792546819434d8bea5f8087d140de as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/5e3792546819434d8bea5f8087d140de 2023-05-02 02:55:31,259 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/5e3792546819434d8bea5f8087d140de, entries=10, sequenceid=19, filesize=5.0 K 2023-05-02 02:55:31,260 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 4b5519600ec1165492215f7e7e85c863 in 36ms, sequenceid=19, compaction requested=false 2023-05-02 02:55:31,260 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 4b5519600ec1165492215f7e7e85c863: 2023-05-02 02:55:31,260 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1682996129638.4b5519600ec1165492215f7e7e85c863. 2023-05-02 02:55:31,260 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-02 02:55:31,260 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-02 02:55:31,260 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,260 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-02 02:55:31,260 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42019,1682996021676' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-02 02:55:31,262 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,262 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:31,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:31,262 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,262 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:31,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:31,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:31,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-02 02:55:31,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:31,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:31,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:31,264 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42019,1682996021676' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-05-02 02:55:31,264 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1b15a465[Count = 0] remaining members to acquire global barrier 2023-05-02 02:55:31,264 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-05-02 02:55:31,264 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,266 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,266 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,266 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,266 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,266 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,266 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-02 02:55:31,266 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,266 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,266 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-02 02:55:31,266 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-02 02:55:31,266 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,42019,1682996021676' in zk 2023-05-02 02:55:31,266 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-02 02:55:31,266 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-02 02:55:31,266 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-02 02:55:31,266 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-02 02:55:31,266 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,38925,1682996021633' in zk 2023-05-02 02:55:31,266 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,39081,1682996021579' in zk 2023-05-02 02:55:31,267 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,267 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-02 02:55:31,267 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:31,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:31,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:31,267 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-02 02:55:31,268 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-02 02:55:31,268 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:31,268 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-02 02:55:31,269 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-02 02:55:31,269 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:31,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:31,269 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-02 02:55:31,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:31,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-02 02:55:31,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:31,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:31,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:31,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-02 02:55:31,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:31,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:31,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,42019,1682996021676': 2023-05-02 02:55:31,272 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42019,1682996021676' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-05-02 02:55:31,272 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-05-02 02:55:31,272 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-02 02:55:31,272 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-02 02:55:31,272 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-05-02 02:55:31,272 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-02 02:55:31,274 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,274 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,274 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,274 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,274 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:31,275 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,275 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:31,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,274 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,274 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,274 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:31,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,275 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:31,275 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:31,275 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:31,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:31,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:31,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:31,275 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:31,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:31,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:31,275 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:31,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,276 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:31,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-02 02:55:31,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:31,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-02 02:55:31,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:31,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,277 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:31,277 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:31,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:31,278 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:31,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-02 02:55:31,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:31,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:31,280 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:31,280 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:31,280 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:31,280 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:31,280 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:31,280 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:31,280 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:31,280 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-05-02 02:55:31,280 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-02 02:55:31,280 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-02 02:55:31,280 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:31,280 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:31,280 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:31,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:31,280 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:31,281 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-05-02 02:55:31,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:31,281 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-02 02:55:31,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:31,280 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,281 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:31,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:31,281 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:31,281 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:31,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:31,281 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-02 02:55:31,281 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:31,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:31,281 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:31,281 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,281 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:31,281 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,281 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,281 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,381 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-02 02:55:31,382 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-02 02:55:31,392 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-02 02:55:31,393 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-05-02 02:55:31,393 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-05-02 02:55:31,393 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-05-02 02:55:31,393 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:31,394 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-05-02 02:55:31,394 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-05-02 02:55:31,394 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,394 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-02 02:55:31,395 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:31,395 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:31,395 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,395 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:31,395 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:31,395 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:31,395 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,396 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-02 02:55:31,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:31,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:31,396 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:31,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:31,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-02 02:55:31,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-02 02:55:31,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-02 02:55:31,396 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,396 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,396 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:31,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:31,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-02 02:55:31,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-02 02:55:31,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:31,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-02 02:55:31,397 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-02 02:55:31,397 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-02 02:55:31,397 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-02 02:55:31,397 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-05-02 02:55:31,397 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-02 02:55:31,397 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-05-02 02:55:31,397 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-02 02:55:31,397 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-05-02 02:55:31,397 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-02 02:55:31,397 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-02 02:55:31,397 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-02 02:55:31,397 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:31,397 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:31,397 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-02 02:55:31,397 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-02 02:55:31,397 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-02 02:55:31,397 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-02 02:55:31,398 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38925,1682996021633' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-02 02:55:31,398 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:31,397 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-02 02:55:31,398 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-02 02:55:31,398 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-02 02:55:31,398 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-02 02:55:31,398 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-02 02:55:31,398 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1682996129638.4b5519600ec1165492215f7e7e85c863. 2023-05-02 02:55:31,398 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1682996129638.4b5519600ec1165492215f7e7e85c863. started... 2023-05-02 02:55:31,398 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 4b5519600ec1165492215f7e7e85c863 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-02 02:55:31,399 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,403 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,403 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,403 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:31,403 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,403 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:31,406 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/.tmp/cf1/4e4d404e9d4a496499b0ada94af7a3ed 2023-05-02 02:55:31,416 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/.tmp/cf2/fdfb5f14b6794ceb90362e1e11a88edf 2023-05-02 02:55:31,419 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/.tmp/cf1/4e4d404e9d4a496499b0ada94af7a3ed as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/4e4d404e9d4a496499b0ada94af7a3ed 2023-05-02 02:55:31,422 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/4e4d404e9d4a496499b0ada94af7a3ed, entries=10, sequenceid=32, filesize=5.0 K 2023-05-02 02:55:31,423 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/.tmp/cf2/fdfb5f14b6794ceb90362e1e11a88edf as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/fdfb5f14b6794ceb90362e1e11a88edf 2023-05-02 02:55:31,426 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/fdfb5f14b6794ceb90362e1e11a88edf, entries=10, sequenceid=32, filesize=5.0 K 2023-05-02 02:55:31,426 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 4b5519600ec1165492215f7e7e85c863 in 28ms, sequenceid=32, compaction requested=true 2023-05-02 02:55:31,426 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 4b5519600ec1165492215f7e7e85c863: 2023-05-02 02:55:31,426 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1682996129638.4b5519600ec1165492215f7e7e85c863. 2023-05-02 02:55:31,427 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-02 02:55:31,427 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-02 02:55:31,427 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,427 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-02 02:55:31,427 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42019,1682996021676' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-02 02:55:31,428 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,428 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:31,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:31,428 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,428 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:31,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:31,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:31,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-02 02:55:31,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:31,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:31,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:31,430 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42019,1682996021676' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-05-02 02:55:31,430 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1d372873[Count = 0] remaining members to acquire global barrier 2023-05-02 02:55:31,430 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-05-02 02:55:31,430 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,431 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,431 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,431 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,431 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,431 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-02 02:55:31,431 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,431 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-02 02:55:31,431 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,431 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,38925,1682996021633' in zk 2023-05-02 02:55:31,431 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-02 02:55:31,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,431 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,431 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-02 02:55:31,431 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-02 02:55:31,431 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,39081,1682996021579' in zk 2023-05-02 02:55:31,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,431 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-02 02:55:31,431 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-02 02:55:31,431 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,42019,1682996021676' in zk 2023-05-02 02:55:31,433 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-02 02:55:31,433 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:31,433 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-02 02:55:31,434 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-02 02:55:31,434 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:31,434 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-02 02:55:31,434 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,434 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-02 02:55:31,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:31,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:31,435 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:31,435 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-02 02:55:31,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:31,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:31,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-02 02:55:31,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:31,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:31,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:31,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-02 02:55:31,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:31,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:31,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,42019,1682996021676': 2023-05-02 02:55:31,438 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42019,1682996021676' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-05-02 02:55:31,438 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-05-02 02:55:31,438 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-02 02:55:31,438 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-02 02:55:31,438 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-05-02 02:55:31,438 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-02 02:55:31,439 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,439 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:31,439 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,439 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,439 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,439 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:31,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:31,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,439 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,439 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:31,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,440 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:31,439 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,439 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:31,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:31,440 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:31,440 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:31,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:31,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,440 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:31,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:31,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:31,441 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-02 02:55:31,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,441 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:31,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:31,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-02 02:55:31,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:31,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:31,442 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:31,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:31,443 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-02 02:55:31,443 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:31,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:31,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:31,447 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:31,447 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:31,447 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:31,447 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:31,447 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:31,447 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:31,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:31,447 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:31,447 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:31,447 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-05-02 02:55:31,447 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-02 02:55:31,447 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:31,447 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-02 02:55:31,447 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:31,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:31,447 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:31,447 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,447 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:31,447 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-05-02 02:55:31,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:31,448 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-02 02:55:31,448 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:31,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:31,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:31,448 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:31,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:31,447 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:31,448 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-02 02:55:31,448 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:31,448 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,448 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,448 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:31,448 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,448 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,548 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-02 02:55:31,549 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-02 02:55:31,559 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-02 02:55:31,561 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-05-02 02:55:31,561 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-05-02 02:55:31,561 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-05-02 02:55:31,561 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:31,561 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-05-02 02:55:31,561 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-05-02 02:55:31,561 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,561 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-02 02:55:31,562 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:31,562 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:31,563 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:31,563 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:31,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:31,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:31,562 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,562 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:31,563 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:31,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:31,563 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,563 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-02 02:55:31,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-02 02:55:31,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-02 02:55:31,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-02 02:55:31,563 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,563 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,563 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:31,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-02 02:55:31,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:31,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:31,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-02 02:55:31,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-02 02:55:31,564 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-02 02:55:31,564 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-02 02:55:31,564 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-02 02:55:31,564 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-05-02 02:55:31,564 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-02 02:55:31,564 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-05-02 02:55:31,564 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-02 02:55:31,564 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:31,564 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-05-02 02:55:31,564 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-02 02:55:31,564 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-02 02:55:31,564 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-02 02:55:31,565 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-02 02:55:31,565 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:31,564 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-02 02:55:31,565 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-02 02:55:31,565 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-02 02:55:31,565 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-02 02:55:31,565 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-02 02:55:31,565 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:31,565 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38925,1682996021633' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-02 02:55:31,565 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-02 02:55:31,565 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-02 02:55:31,565 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-02 02:55:31,565 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1682996129638.4b5519600ec1165492215f7e7e85c863. 2023-05-02 02:55:31,565 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1682996129638.4b5519600ec1165492215f7e7e85c863. started... 2023-05-02 02:55:31,565 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 4b5519600ec1165492215f7e7e85c863 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-02 02:55:31,566 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,567 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,567 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,567 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:31,567 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,567 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:31,577 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/.tmp/cf1/3e1e2cafee0643c6815088f36cad45af 2023-05-02 02:55:31,588 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/.tmp/cf2/c030ba5f846346de9dea5f5911f62fe6 2023-05-02 02:55:31,592 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/.tmp/cf1/3e1e2cafee0643c6815088f36cad45af as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/3e1e2cafee0643c6815088f36cad45af 2023-05-02 02:55:31,595 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/3e1e2cafee0643c6815088f36cad45af, entries=10, sequenceid=45, filesize=5.0 K 2023-05-02 02:55:31,595 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/.tmp/cf2/c030ba5f846346de9dea5f5911f62fe6 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/c030ba5f846346de9dea5f5911f62fe6 2023-05-02 02:55:31,598 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/c030ba5f846346de9dea5f5911f62fe6, entries=10, sequenceid=45, filesize=5.0 K 2023-05-02 02:55:31,599 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 4b5519600ec1165492215f7e7e85c863 in 34ms, sequenceid=45, compaction requested=true 2023-05-02 02:55:31,599 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 4b5519600ec1165492215f7e7e85c863: 2023-05-02 02:55:31,599 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1682996129638.4b5519600ec1165492215f7e7e85c863. 2023-05-02 02:55:31,599 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-02 02:55:31,599 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-02 02:55:31,599 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,599 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-02 02:55:31,599 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42019,1682996021676' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-02 02:55:31,602 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,602 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:31,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:31,602 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,602 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:31,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:31,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:31,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-02 02:55:31,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:31,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:31,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:31,604 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42019,1682996021676' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-05-02 02:55:31,604 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1d0653fb[Count = 0] remaining members to acquire global barrier 2023-05-02 02:55:31,604 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-05-02 02:55:31,604 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,605 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,605 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,605 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,605 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,605 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,605 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-02 02:55:31,605 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-02 02:55:31,605 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,605 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,39081,1682996021579' in zk 2023-05-02 02:55:31,605 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,605 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-02 02:55:31,605 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-02 02:55:31,605 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-02 02:55:31,605 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,38925,1682996021633' in zk 2023-05-02 02:55:31,605 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-02 02:55:31,606 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-02 02:55:31,606 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,42019,1682996021676' in zk 2023-05-02 02:55:31,606 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-02 02:55:31,606 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:31,606 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-02 02:55:31,607 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-02 02:55:31,607 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:31,607 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-02 02:55:31,609 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:31,609 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-02 02:55:31,609 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:31,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:31,609 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-02 02:55:31,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:31,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:31,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-02 02:55:31,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:31,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:31,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:31,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-02 02:55:31,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:31,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:31,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,42019,1682996021676': 2023-05-02 02:55:31,612 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42019,1682996021676' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-05-02 02:55:31,612 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-05-02 02:55:31,612 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-02 02:55:31,612 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-02 02:55:31,612 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-05-02 02:55:31,612 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-02 02:55:31,613 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,613 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,613 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,613 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,613 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:31,613 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:31,613 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,613 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,613 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:31,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:31,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:31,613 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,614 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:31,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,614 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:31,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:31,614 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:31,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:31,614 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:31,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:31,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:31,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,615 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-02 02:55:31,615 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:31,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:31,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-02 02:55:31,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:31,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:31,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:31,617 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:31,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-02 02:55:31,617 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:31,618 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:31,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:31,623 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:31,623 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-02 02:55:31,623 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:31,623 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,623 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:31,623 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:31,623 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:31,623 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:31,623 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:31,623 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:31,623 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:31,623 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:31,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:31,623 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:31,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:31,623 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-02 02:55:31,623 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:31,624 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:31,623 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-05-02 02:55:31,624 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-02 02:55:31,624 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:31,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:31,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:31,624 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,624 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,624 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:31,624 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:31,624 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-05-02 02:55:31,624 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:31,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:31,624 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,625 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:31,624 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-02 02:55:31,725 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-02 02:55:31,726 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-02 02:55:31,736 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-02 02:55:31,737 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-05-02 02:55:31,737 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-05-02 02:55:31,737 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-05-02 02:55:31,737 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:31,737 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-05-02 02:55:31,737 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-05-02 02:55:31,738 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,738 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-02 02:55:31,739 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,739 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:31,739 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:31,739 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:31,739 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:31,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:31,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:31,739 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,739 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-02 02:55:31,739 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:31,739 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:31,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:31,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-02 02:55:31,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-02 02:55:31,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-02 02:55:31,740 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,740 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,740 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:31,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:31,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-02 02:55:31,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-02 02:55:31,740 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-02 02:55:31,740 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-02 02:55:31,741 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-05-02 02:55:31,741 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-02 02:55:31,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:31,741 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-02 02:55:31,741 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-05-02 02:55:31,741 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:31,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-02 02:55:31,741 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-02 02:55:31,741 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-02 02:55:31,741 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-02 02:55:31,741 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-02 02:55:31,741 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:31,741 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-02 02:55:31,741 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-05-02 02:55:31,741 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1682996129638.4b5519600ec1165492215f7e7e85c863. 2023-05-02 02:55:31,741 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-02 02:55:31,741 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-02 02:55:31,741 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38925,1682996021633' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-02 02:55:31,741 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1682996129638.4b5519600ec1165492215f7e7e85c863. started... 2023-05-02 02:55:31,741 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-02 02:55:31,741 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-02 02:55:31,742 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:31,742 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 4b5519600ec1165492215f7e7e85c863 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-02 02:55:31,741 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-02 02:55:31,742 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-02 02:55:31,742 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-02 02:55:31,742 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-02 02:55:31,743 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,744 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,744 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,744 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:31,744 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,744 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:31,750 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/.tmp/cf1/988f32ec168a465cb6ca4435884e9032 2023-05-02 02:55:31,761 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/.tmp/cf2/92d58d8357e44ee8901c47dbe21bc584 2023-05-02 02:55:31,765 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/.tmp/cf1/988f32ec168a465cb6ca4435884e9032 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/988f32ec168a465cb6ca4435884e9032 2023-05-02 02:55:31,768 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/988f32ec168a465cb6ca4435884e9032, entries=10, sequenceid=58, filesize=5.0 K 2023-05-02 02:55:31,769 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/.tmp/cf2/92d58d8357e44ee8901c47dbe21bc584 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/92d58d8357e44ee8901c47dbe21bc584 2023-05-02 02:55:31,772 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/92d58d8357e44ee8901c47dbe21bc584, entries=10, sequenceid=58, filesize=5.0 K 2023-05-02 02:55:31,773 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 4b5519600ec1165492215f7e7e85c863 in 32ms, sequenceid=58, compaction requested=true 2023-05-02 02:55:31,773 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 4b5519600ec1165492215f7e7e85c863: 2023-05-02 02:55:31,773 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1682996129638.4b5519600ec1165492215f7e7e85c863. 2023-05-02 02:55:31,773 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-02 02:55:31,773 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-02 02:55:31,773 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,773 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-02 02:55:31,773 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42019,1682996021676' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-02 02:55:31,775 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,775 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:31,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:31,775 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,775 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:31,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:31,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:31,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-02 02:55:31,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:31,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:31,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:31,776 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42019,1682996021676' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-05-02 02:55:31,776 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2ecdd423[Count = 0] remaining members to acquire global barrier 2023-05-02 02:55:31,776 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-05-02 02:55:31,777 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,777 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,777 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,777 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,777 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,777 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,778 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,778 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,778 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-02 02:55:31,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,778 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-02 02:55:31,778 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-02 02:55:31,778 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-02 02:55:31,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,778 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,39081,1682996021579' in zk 2023-05-02 02:55:31,778 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-02 02:55:31,778 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-02 02:55:31,778 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-02 02:55:31,778 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,42019,1682996021676' in zk 2023-05-02 02:55:31,778 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,38925,1682996021633' in zk 2023-05-02 02:55:31,779 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-02 02:55:31,779 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:31,779 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-02 02:55:31,780 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,780 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-02 02:55:31,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:31,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:31,780 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:31,780 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-02 02:55:31,781 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:31,781 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-02 02:55:31,781 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-02 02:55:31,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:31,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:31,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-02 02:55:31,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:31,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:31,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:31,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-02 02:55:31,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:31,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:31,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,42019,1682996021676': 2023-05-02 02:55:31,783 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42019,1682996021676' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-05-02 02:55:31,783 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-05-02 02:55:31,783 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-02 02:55:31,783 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-02 02:55:31,783 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-05-02 02:55:31,783 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-02 02:55:31,784 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,784 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,785 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,785 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:31,785 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:31,785 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:31,785 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:31,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:31,785 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,785 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:31,785 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:31,785 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:31,785 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:31,785 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:31,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:31,786 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-02 02:55:31,786 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:31,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:31,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,786 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:31,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:31,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-02 02:55:31,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:31,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:31,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:31,788 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:31,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-02 02:55:31,788 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:31,788 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:31,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:31,791 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:31,791 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:31,791 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:31,791 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:31,791 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:31,791 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:31,791 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:31,791 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:31,791 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:31,791 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:31,791 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:31,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:31,792 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-02 02:55:31,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:31,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:31,791 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-05-02 02:55:31,792 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-02 02:55:31,792 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,792 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:31,792 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-02 02:55:31,792 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:31,792 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:31,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:31,792 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:31,792 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:31,792 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,792 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:31,792 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,792 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,792 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-05-02 02:55:31,792 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-02 02:55:31,792 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:31,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:31,893 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-02 02:55:31,894 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-02 02:55:31,904 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-02 02:55:31,906 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-05-02 02:55:31,906 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-05-02 02:55:31,906 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-05-02 02:55:31,906 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:31,906 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-05-02 02:55:31,906 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-05-02 02:55:31,906 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,906 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-02 02:55:31,908 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:31,908 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:31,908 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,908 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:31,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:31,908 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:31,908 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:31,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:31,908 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:31,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:31,908 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,908 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-02 02:55:31,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-02 02:55:31,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-02 02:55:31,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-02 02:55:31,909 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,909 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,909 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:31,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:31,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-02 02:55:31,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-02 02:55:31,909 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-02 02:55:31,909 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-02 02:55:31,909 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-05-02 02:55:31,909 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-02 02:55:31,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-02 02:55:31,909 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-02 02:55:31,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-02 02:55:31,909 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-05-02 02:55:31,909 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-02 02:55:31,909 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:31,910 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-02 02:55:31,910 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-02 02:55:31,910 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-02 02:55:31,910 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:31,910 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-02 02:55:31,910 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-02 02:55:31,910 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-02 02:55:31,910 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38925,1682996021633' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-02 02:55:31,910 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-02 02:55:31,910 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-05-02 02:55:31,910 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1682996129638.4b5519600ec1165492215f7e7e85c863. 2023-05-02 02:55:31,910 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1682996129638.4b5519600ec1165492215f7e7e85c863. started... 2023-05-02 02:55:31,910 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-02 02:55:31,910 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-02 02:55:31,910 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 4b5519600ec1165492215f7e7e85c863 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-02 02:55:31,910 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-02 02:55:31,910 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-02 02:55:31,910 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-02 02:55:31,910 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39081,1682996021579' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-02 02:55:31,912 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,913 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,913 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,913 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:31,914 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,914 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:31,922 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/.tmp/cf1/5c333cfab2c447638884f21583402bdd 2023-05-02 02:55:31,938 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/.tmp/cf2/0900e2b6e1284554a0808b2855de8862 2023-05-02 02:55:31,941 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/.tmp/cf1/5c333cfab2c447638884f21583402bdd as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/5c333cfab2c447638884f21583402bdd 2023-05-02 02:55:31,944 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/5c333cfab2c447638884f21583402bdd, entries=10, sequenceid=71, filesize=5.0 K 2023-05-02 02:55:31,945 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/.tmp/cf2/0900e2b6e1284554a0808b2855de8862 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/0900e2b6e1284554a0808b2855de8862 2023-05-02 02:55:31,948 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/0900e2b6e1284554a0808b2855de8862, entries=10, sequenceid=71, filesize=5.0 K 2023-05-02 02:55:31,949 INFO [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 4b5519600ec1165492215f7e7e85c863 in 39ms, sequenceid=71, compaction requested=true 2023-05-02 02:55:31,949 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 4b5519600ec1165492215f7e7e85c863: 2023-05-02 02:55:31,949 DEBUG [rs(jenkins-hbase4.apache.org,42019,1682996021676)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1682996129638.4b5519600ec1165492215f7e7e85c863. 2023-05-02 02:55:31,949 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-02 02:55:31,949 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-02 02:55:31,949 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,949 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-02 02:55:31,949 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42019,1682996021676' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-02 02:55:31,951 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,951 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:31,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:31,951 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,951 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-02 02:55:31,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:31,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:31,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-02 02:55:31,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:31,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:31,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:31,953 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42019,1682996021676' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-05-02 02:55:31,953 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@184d7b13[Count = 0] remaining members to acquire global barrier 2023-05-02 02:55:31,953 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-05-02 02:55:31,953 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,955 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,955 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,955 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,955 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,955 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,955 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,955 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-02 02:55:31,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,955 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,955 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-02 02:55:31,955 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-02 02:55:31,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,955 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,42019,1682996021676' in zk 2023-05-02 02:55:31,955 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-02 02:55:31,955 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-02 02:55:31,955 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-02 02:55:31,955 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,39081,1682996021579' in zk 2023-05-02 02:55:31,955 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-02 02:55:31,955 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,38925,1682996021633' in zk 2023-05-02 02:55:31,965 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-02 02:55:31,965 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,965 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:31,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:31,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:31,965 DEBUG [member: 'jenkins-hbase4.apache.org,42019,1682996021676' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-02 02:55:31,965 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-02 02:55:31,965 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-02 02:55:31,965 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:31,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:31,965 DEBUG [member: 'jenkins-hbase4.apache.org,39081,1682996021579' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-02 02:55:31,965 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:31,966 DEBUG [member: 'jenkins-hbase4.apache.org,38925,1682996021633' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-02 02:55:31,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:31,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-02 02:55:31,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:31,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:31,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:31,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-02 02:55:31,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:31,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:31,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,42019,1682996021676': 2023-05-02 02:55:31,968 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42019,1682996021676' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-05-02 02:55:31,968 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-05-02 02:55:31,969 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-02 02:55:31,969 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-02 02:55:31,969 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-05-02 02:55:31,969 INFO [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-02 02:55:31,970 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,970 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-02 02:55:31,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-02 02:55:31,970 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,970 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,970 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,970 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:31,970 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:31,970 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,971 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:31,970 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:31,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,970 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:31,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-02 02:55:31,971 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:31,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,971 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:31,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:31,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-02 02:55:31,971 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:31,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:31,971 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-02 02:55:31,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,972 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:31,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-02 02:55:31,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:31,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:31,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-02 02:55:31,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-02 02:55:31,974 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,974 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:31,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:31,974 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:31,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:31,977 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:31,977 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:31,977 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-02 02:55:31,978 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:31,978 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:31,978 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:31,978 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:31,978 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-02 02:55:31,978 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:31,977 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-02 02:55:31,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:31,977 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-02 02:55:31,978 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,978 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-05-02 02:55:31,978 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-02 02:55:31,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:31,978 DEBUG [(jenkins-hbase4.apache.org,41119,1682996020397)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-02 02:55:31,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-02 02:55:31,978 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:31,978 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-02 02:55:31,978 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:31,978 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,978 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:31,978 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:31,978 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-02 02:55:31,978 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:31,978 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-05-02 02:55:31,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:31,978 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-02 02:55:31,979 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:31,979 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-02 02:55:31,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:31,979 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-02 02:55:31,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-02 02:55:32,079 DEBUG [Listener at localhost/37119] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-02 02:55:32,080 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-02 02:55:32,087 DEBUG [Listener at localhost/37119] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:32,088 DEBUG [Listener at localhost/37119] procedure2.ProcedureExecutor(1029): Stored pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=4b5519600ec1165492215f7e7e85c863, daughterA=ba9cdc1b6bbd97ecf605751816de6a67, daughterB=14a0d63ed7997d8b7bc0247c16a8acc1 2023-05-02 02:55:32,089 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=4b5519600ec1165492215f7e7e85c863, daughterA=ba9cdc1b6bbd97ecf605751816de6a67, daughterB=14a0d63ed7997d8b7bc0247c16a8acc1 2023-05-02 02:55:32,089 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=4b5519600ec1165492215f7e7e85c863, daughterA=ba9cdc1b6bbd97ecf605751816de6a67, daughterB=14a0d63ed7997d8b7bc0247c16a8acc1 2023-05-02 02:55:32,089 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=4b5519600ec1165492215f7e7e85c863, daughterA=ba9cdc1b6bbd97ecf605751816de6a67, daughterB=14a0d63ed7997d8b7bc0247c16a8acc1 2023-05-02 02:55:32,090 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=4b5519600ec1165492215f7e7e85c863, UNASSIGN}] 2023-05-02 02:55:32,091 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=4b5519600ec1165492215f7e7e85c863, UNASSIGN 2023-05-02 02:55:32,092 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=4b5519600ec1165492215f7e7e85c863, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:32,092 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1682996129638.4b5519600ec1165492215f7e7e85c863.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1682996132092"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1682996132092"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682996132092"}]},"ts":"1682996132092"} 2023-05-02 02:55:32,093 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=146, ppid=145, state=RUNNABLE; CloseRegionProcedure 4b5519600ec1165492215f7e7e85c863, server=jenkins-hbase4.apache.org,42019,1682996021676}] 2023-05-02 02:55:32,243 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:32,243 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 4b5519600ec1165492215f7e7e85c863, disabling compactions & flushes 2023-05-02 02:55:32,243 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegion,,1682996129638.4b5519600ec1165492215f7e7e85c863. 2023-05-02 02:55:32,244 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegion,,1682996129638.4b5519600ec1165492215f7e7e85c863. 2023-05-02 02:55:32,244 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegion,,1682996129638.4b5519600ec1165492215f7e7e85c863. after waiting 0 ms 2023-05-02 02:55:32,244 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegion,,1682996129638.4b5519600ec1165492215f7e7e85c863. 2023-05-02 02:55:32,244 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 4b5519600ec1165492215f7e7e85c863 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-05-02 02:55:32,252 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/.tmp/cf1/12156fd9614f4ac59e717b6195f80411 2023-05-02 02:55:32,263 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/.tmp/cf2/1c974c743be54651beb4e762e81df9c3 2023-05-02 02:55:32,266 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/.tmp/cf1/12156fd9614f4ac59e717b6195f80411 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/12156fd9614f4ac59e717b6195f80411 2023-05-02 02:55:32,269 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/12156fd9614f4ac59e717b6195f80411, entries=8, sequenceid=82, filesize=5.0 K 2023-05-02 02:55:32,270 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/.tmp/cf2/1c974c743be54651beb4e762e81df9c3 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/1c974c743be54651beb4e762e81df9c3 2023-05-02 02:55:32,273 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/1c974c743be54651beb4e762e81df9c3, entries=8, sequenceid=82, filesize=5.0 K 2023-05-02 02:55:32,274 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 4b5519600ec1165492215f7e7e85c863 in 30ms, sequenceid=82, compaction requested=true 2023-05-02 02:55:32,279 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-05-02 02:55:32,279 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-02 02:55:32,280 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegion,,1682996129638.4b5519600ec1165492215f7e7e85c863. 2023-05-02 02:55:32,280 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 4b5519600ec1165492215f7e7e85c863: 2023-05-02 02:55:32,281 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:32,281 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=4b5519600ec1165492215f7e7e85c863, regionState=CLOSED 2023-05-02 02:55:32,281 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1682996129638.4b5519600ec1165492215f7e7e85c863.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1682996132281"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682996132281"}]},"ts":"1682996132281"} 2023-05-02 02:55:32,284 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=146, resume processing ppid=145 2023-05-02 02:55:32,284 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=146, ppid=145, state=SUCCESS; CloseRegionProcedure 4b5519600ec1165492215f7e7e85c863, server=jenkins-hbase4.apache.org,42019,1682996021676 in 189 msec 2023-05-02 02:55:32,285 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=145, resume processing ppid=144 2023-05-02 02:55:32,285 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=145, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=4b5519600ec1165492215f7e7e85c863, UNASSIGN in 194 msec 2023-05-02 02:55:32,293 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=144 splitting 14 storefiles, region=4b5519600ec1165492215f7e7e85c863, threads=14 2023-05-02 02:55:32,294 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/0900e2b6e1284554a0808b2855de8862 for region: 4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:32,294 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/1c974c743be54651beb4e762e81df9c3 for region: 4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:32,294 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/5e3792546819434d8bea5f8087d140de for region: 4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:32,294 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/92d58d8357e44ee8901c47dbe21bc584 for region: 4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:32,294 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/b4e5a86ddabc457190064c61898d51ca for region: 4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:32,294 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/c030ba5f846346de9dea5f5911f62fe6 for region: 4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:32,294 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/fdfb5f14b6794ceb90362e1e11a88edf for region: 4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:32,294 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/12156fd9614f4ac59e717b6195f80411 for region: 4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:32,295 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/3e1e2cafee0643c6815088f36cad45af for region: 4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:32,295 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/40c93f88f8df4f8e85dfd0fe36da6ea3 for region: 4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:32,295 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/4e4d404e9d4a496499b0ada94af7a3ed for region: 4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:32,295 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/5c333cfab2c447638884f21583402bdd for region: 4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:32,295 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/a15ebfd300ef4d08a9526159dd2f49ab for region: 4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:32,295 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/988f32ec168a465cb6ca4435884e9032 for region: 4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:32,456 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/0900e2b6e1284554a0808b2855de8862 for region: 4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:32,464 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/5e3792546819434d8bea5f8087d140de for region: 4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:32,465 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/a15ebfd300ef4d08a9526159dd2f49ab for region: 4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:32,488 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/1c974c743be54651beb4e762e81df9c3 for region: 4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:32,505 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/c030ba5f846346de9dea5f5911f62fe6 for region: 4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:32,508 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/988f32ec168a465cb6ca4435884e9032 for region: 4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:32,512 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/5c333cfab2c447638884f21583402bdd for region: 4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:32,524 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/12156fd9614f4ac59e717b6195f80411 for region: 4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:32,524 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/3e1e2cafee0643c6815088f36cad45af for region: 4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:32,525 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/4e4d404e9d4a496499b0ada94af7a3ed for region: 4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:32,534 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/92d58d8357e44ee8901c47dbe21bc584 for region: 4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:32,535 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/fdfb5f14b6794ceb90362e1e11a88edf for region: 4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:32,535 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/40c93f88f8df4f8e85dfd0fe36da6ea3 for region: 4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:32,537 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/b4e5a86ddabc457190064c61898d51ca for region: 4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:32,537 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=144 split storefiles for region 4b5519600ec1165492215f7e7e85c863 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-05-02 02:55:32,566 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-05-02 02:55:32,567 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-05-02 02:55:32,569 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1682996129638.4b5519600ec1165492215f7e7e85c863.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1682996132569"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1682996132569"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1682996132569"}]},"ts":"1682996132569"} 2023-05-02 02:55:32,569 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1682996132087.ba9cdc1b6bbd97ecf605751816de6a67.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1682996132569"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682996132569"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682996132569"}]},"ts":"1682996132569"} 2023-05-02 02:55:32,569 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1682996132087.14a0d63ed7997d8b7bc0247c16a8acc1.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1682996132569"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682996132569"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682996132569"}]},"ts":"1682996132569"} 2023-05-02 02:55:32,571 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=ba9cdc1b6bbd97ecf605751816de6a67, ASSIGN}, {pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=14a0d63ed7997d8b7bc0247c16a8acc1, ASSIGN}] 2023-05-02 02:55:32,572 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=14a0d63ed7997d8b7bc0247c16a8acc1, ASSIGN 2023-05-02 02:55:32,572 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=14a0d63ed7997d8b7bc0247c16a8acc1, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,42019,1682996021676; forceNewPlan=false, retain=false 2023-05-02 02:55:32,573 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=ba9cdc1b6bbd97ecf605751816de6a67, ASSIGN 2023-05-02 02:55:32,573 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=ba9cdc1b6bbd97ecf605751816de6a67, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,42019,1682996021676; forceNewPlan=false, retain=false 2023-05-02 02:55:32,723 INFO [jenkins-hbase4:41119] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-05-02 02:55:32,725 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=ba9cdc1b6bbd97ecf605751816de6a67, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:32,725 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1682996132087.ba9cdc1b6bbd97ecf605751816de6a67.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1682996132725"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1682996132725"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682996132725"}]},"ts":"1682996132725"} 2023-05-02 02:55:32,727 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=149, ppid=147, state=RUNNABLE; OpenRegionProcedure ba9cdc1b6bbd97ecf605751816de6a67, server=jenkins-hbase4.apache.org,42019,1682996021676}] 2023-05-02 02:55:32,729 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=14a0d63ed7997d8b7bc0247c16a8acc1, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:32,729 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1682996132087.14a0d63ed7997d8b7bc0247c16a8acc1.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1682996132729"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1682996132729"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682996132729"}]},"ts":"1682996132729"} 2023-05-02 02:55:32,730 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=150, ppid=148, state=RUNNABLE; OpenRegionProcedure 14a0d63ed7997d8b7bc0247c16a8acc1, server=jenkins-hbase4.apache.org,42019,1682996021676}] 2023-05-02 02:55:32,881 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegion,41,1682996132087.14a0d63ed7997d8b7bc0247c16a8acc1. 2023-05-02 02:55:32,881 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 14a0d63ed7997d8b7bc0247c16a8acc1, NAME => 'testSplitTableRegion,41,1682996132087.14a0d63ed7997d8b7bc0247c16a8acc1.', STARTKEY => '41', ENDKEY => ''} 2023-05-02 02:55:32,881 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-02 02:55:32,881 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 14a0d63ed7997d8b7bc0247c16a8acc1 2023-05-02 02:55:32,881 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegion,41,1682996132087.14a0d63ed7997d8b7bc0247c16a8acc1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-02 02:55:32,882 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 14a0d63ed7997d8b7bc0247c16a8acc1 2023-05-02 02:55:32,882 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 14a0d63ed7997d8b7bc0247c16a8acc1 2023-05-02 02:55:32,882 INFO [StoreOpener-14a0d63ed7997d8b7bc0247c16a8acc1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 14a0d63ed7997d8b7bc0247c16a8acc1 2023-05-02 02:55:32,883 DEBUG [StoreOpener-14a0d63ed7997d8b7bc0247c16a8acc1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf1 2023-05-02 02:55:32,883 DEBUG [StoreOpener-14a0d63ed7997d8b7bc0247c16a8acc1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf1 2023-05-02 02:55:32,884 INFO [StoreOpener-14a0d63ed7997d8b7bc0247c16a8acc1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 14a0d63ed7997d8b7bc0247c16a8acc1 columnFamilyName cf1 2023-05-02 02:55:32,896 DEBUG [StoreOpener-14a0d63ed7997d8b7bc0247c16a8acc1-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf1/12156fd9614f4ac59e717b6195f80411.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/12156fd9614f4ac59e717b6195f80411-top 2023-05-02 02:55:32,899 DEBUG [StoreOpener-14a0d63ed7997d8b7bc0247c16a8acc1-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf1/3e1e2cafee0643c6815088f36cad45af.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/3e1e2cafee0643c6815088f36cad45af-top 2023-05-02 02:55:32,902 DEBUG [StoreOpener-14a0d63ed7997d8b7bc0247c16a8acc1-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf1/40c93f88f8df4f8e85dfd0fe36da6ea3.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/40c93f88f8df4f8e85dfd0fe36da6ea3-top 2023-05-02 02:55:32,905 DEBUG [StoreOpener-14a0d63ed7997d8b7bc0247c16a8acc1-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf1/4e4d404e9d4a496499b0ada94af7a3ed.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/4e4d404e9d4a496499b0ada94af7a3ed-top 2023-05-02 02:55:32,908 DEBUG [StoreOpener-14a0d63ed7997d8b7bc0247c16a8acc1-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf1/5c333cfab2c447638884f21583402bdd.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/5c333cfab2c447638884f21583402bdd-top 2023-05-02 02:55:32,911 DEBUG [StoreOpener-14a0d63ed7997d8b7bc0247c16a8acc1-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf1/988f32ec168a465cb6ca4435884e9032.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/988f32ec168a465cb6ca4435884e9032-top 2023-05-02 02:55:32,914 DEBUG [StoreOpener-14a0d63ed7997d8b7bc0247c16a8acc1-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf1/a15ebfd300ef4d08a9526159dd2f49ab.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/a15ebfd300ef4d08a9526159dd2f49ab-top 2023-05-02 02:55:32,914 INFO [StoreOpener-14a0d63ed7997d8b7bc0247c16a8acc1-1] regionserver.HStore(310): Store=14a0d63ed7997d8b7bc0247c16a8acc1/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-02 02:55:32,915 INFO [StoreOpener-14a0d63ed7997d8b7bc0247c16a8acc1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 14a0d63ed7997d8b7bc0247c16a8acc1 2023-05-02 02:55:32,916 DEBUG [StoreOpener-14a0d63ed7997d8b7bc0247c16a8acc1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf2 2023-05-02 02:55:32,916 DEBUG [StoreOpener-14a0d63ed7997d8b7bc0247c16a8acc1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf2 2023-05-02 02:55:32,916 INFO [StoreOpener-14a0d63ed7997d8b7bc0247c16a8acc1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 14a0d63ed7997d8b7bc0247c16a8acc1 columnFamilyName cf2 2023-05-02 02:55:32,928 DEBUG [StoreOpener-14a0d63ed7997d8b7bc0247c16a8acc1-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf2/0900e2b6e1284554a0808b2855de8862.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/0900e2b6e1284554a0808b2855de8862-top 2023-05-02 02:55:32,931 DEBUG [StoreOpener-14a0d63ed7997d8b7bc0247c16a8acc1-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf2/1c974c743be54651beb4e762e81df9c3.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/1c974c743be54651beb4e762e81df9c3-top 2023-05-02 02:55:32,934 DEBUG [StoreOpener-14a0d63ed7997d8b7bc0247c16a8acc1-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf2/5e3792546819434d8bea5f8087d140de.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/5e3792546819434d8bea5f8087d140de-top 2023-05-02 02:55:32,937 DEBUG [StoreOpener-14a0d63ed7997d8b7bc0247c16a8acc1-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf2/92d58d8357e44ee8901c47dbe21bc584.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/92d58d8357e44ee8901c47dbe21bc584-top 2023-05-02 02:55:32,941 DEBUG [StoreOpener-14a0d63ed7997d8b7bc0247c16a8acc1-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf2/b4e5a86ddabc457190064c61898d51ca.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/b4e5a86ddabc457190064c61898d51ca-top 2023-05-02 02:55:32,944 DEBUG [StoreOpener-14a0d63ed7997d8b7bc0247c16a8acc1-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf2/c030ba5f846346de9dea5f5911f62fe6.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/c030ba5f846346de9dea5f5911f62fe6-top 2023-05-02 02:55:32,947 DEBUG [StoreOpener-14a0d63ed7997d8b7bc0247c16a8acc1-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf2/fdfb5f14b6794ceb90362e1e11a88edf.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/fdfb5f14b6794ceb90362e1e11a88edf-top 2023-05-02 02:55:32,947 INFO [StoreOpener-14a0d63ed7997d8b7bc0247c16a8acc1-1] regionserver.HStore(310): Store=14a0d63ed7997d8b7bc0247c16a8acc1/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-02 02:55:32,948 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1 2023-05-02 02:55:32,949 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1 2023-05-02 02:55:32,950 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-02 02:55:32,951 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 14a0d63ed7997d8b7bc0247c16a8acc1 2023-05-02 02:55:32,952 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 14a0d63ed7997d8b7bc0247c16a8acc1; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61203652, jitterRate=-0.0879945158958435}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-02 02:55:32,952 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 14a0d63ed7997d8b7bc0247c16a8acc1: 2023-05-02 02:55:32,953 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,41,1682996132087.14a0d63ed7997d8b7bc0247c16a8acc1., pid=150, masterSystemTime=1682996132878 2023-05-02 02:55:32,953 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-02 02:55:32,954 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-02 02:55:32,954 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-02 02:55:32,954 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-02 02:55:32,954 INFO [RS:2;jenkins-hbase4:42019-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,41,1682996132087.14a0d63ed7997d8b7bc0247c16a8acc1. 2023-05-02 02:55:32,954 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] regionserver.HStore(1912): 14a0d63ed7997d8b7bc0247c16a8acc1/cf2 is initiating minor compaction (all files) 2023-05-02 02:55:32,954 INFO [RS:2;jenkins-hbase4:42019-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,41,1682996132087.14a0d63ed7997d8b7bc0247c16a8acc1. 2023-05-02 02:55:32,954 INFO [RS:2;jenkins-hbase4:42019-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 14a0d63ed7997d8b7bc0247c16a8acc1/cf2 in testSplitTableRegion,41,1682996132087.14a0d63ed7997d8b7bc0247c16a8acc1. 2023-05-02 02:55:32,954 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] regionserver.HStore(1912): 14a0d63ed7997d8b7bc0247c16a8acc1/cf1 is initiating minor compaction (all files) 2023-05-02 02:55:32,954 INFO [RS:2;jenkins-hbase4:42019-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 14a0d63ed7997d8b7bc0247c16a8acc1/cf1 in testSplitTableRegion,41,1682996132087.14a0d63ed7997d8b7bc0247c16a8acc1. 2023-05-02 02:55:32,954 INFO [RS:2;jenkins-hbase4:42019-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf2/b4e5a86ddabc457190064c61898d51ca.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/b4e5a86ddabc457190064c61898d51ca-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf2/5e3792546819434d8bea5f8087d140de.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/5e3792546819434d8bea5f8087d140de-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf2/fdfb5f14b6794ceb90362e1e11a88edf.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/fdfb5f14b6794ceb90362e1e11a88edf-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf2/c030ba5f846346de9dea5f5911f62fe6.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/c030ba5f846346de9dea5f5911f62fe6-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf2/92d58d8357e44ee8901c47dbe21bc584.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/92d58d8357e44ee8901c47dbe21bc584-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf2/0900e2b6e1284554a0808b2855de8862.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/0900e2b6e1284554a0808b2855de8862-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf2/1c974c743be54651beb4e762e81df9c3.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/1c974c743be54651beb4e762e81df9c3-top] into tmpdir=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/.tmp, totalSize=34.9 K 2023-05-02 02:55:32,955 INFO [RS:2;jenkins-hbase4:42019-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf1/40c93f88f8df4f8e85dfd0fe36da6ea3.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/40c93f88f8df4f8e85dfd0fe36da6ea3-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf1/a15ebfd300ef4d08a9526159dd2f49ab.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/a15ebfd300ef4d08a9526159dd2f49ab-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf1/4e4d404e9d4a496499b0ada94af7a3ed.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/4e4d404e9d4a496499b0ada94af7a3ed-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf1/3e1e2cafee0643c6815088f36cad45af.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/3e1e2cafee0643c6815088f36cad45af-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf1/988f32ec168a465cb6ca4435884e9032.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/988f32ec168a465cb6ca4435884e9032-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf1/5c333cfab2c447638884f21583402bdd.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/5c333cfab2c447638884f21583402bdd-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf1/12156fd9614f4ac59e717b6195f80411.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/12156fd9614f4ac59e717b6195f80411-top] into tmpdir=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/.tmp, totalSize=34.9 K 2023-05-02 02:55:32,955 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] compactions.Compactor(207): Compacting b4e5a86ddabc457190064c61898d51ca.4b5519600ec1165492215f7e7e85c863, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1682996130641 2023-05-02 02:55:32,955 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] compactions.Compactor(207): Compacting 40c93f88f8df4f8e85dfd0fe36da6ea3.4b5519600ec1165492215f7e7e85c863, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1682996130641 2023-05-02 02:55:32,956 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,41,1682996132087.14a0d63ed7997d8b7bc0247c16a8acc1. 2023-05-02 02:55:32,956 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegion,41,1682996132087.14a0d63ed7997d8b7bc0247c16a8acc1. 2023-05-02 02:55:32,956 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] compactions.Compactor(207): Compacting 5e3792546819434d8bea5f8087d140de.4b5519600ec1165492215f7e7e85c863, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1682996131209 2023-05-02 02:55:32,956 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegion,,1682996132087.ba9cdc1b6bbd97ecf605751816de6a67. 2023-05-02 02:55:32,956 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=14a0d63ed7997d8b7bc0247c16a8acc1, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:32,956 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => ba9cdc1b6bbd97ecf605751816de6a67, NAME => 'testSplitTableRegion,,1682996132087.ba9cdc1b6bbd97ecf605751816de6a67.', STARTKEY => '', ENDKEY => '41'} 2023-05-02 02:55:32,956 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,41,1682996132087.14a0d63ed7997d8b7bc0247c16a8acc1.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1682996132956"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1682996132956"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1682996132956"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682996132956"}]},"ts":"1682996132956"} 2023-05-02 02:55:32,956 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] compactions.Compactor(207): Compacting a15ebfd300ef4d08a9526159dd2f49ab.4b5519600ec1165492215f7e7e85c863, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1682996131209 2023-05-02 02:55:32,956 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-02 02:55:32,956 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion ba9cdc1b6bbd97ecf605751816de6a67 2023-05-02 02:55:32,957 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegion,,1682996132087.ba9cdc1b6bbd97ecf605751816de6a67.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-02 02:55:32,957 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] compactions.Compactor(207): Compacting 4e4d404e9d4a496499b0ada94af7a3ed.4b5519600ec1165492215f7e7e85c863, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1682996131382 2023-05-02 02:55:32,957 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for ba9cdc1b6bbd97ecf605751816de6a67 2023-05-02 02:55:32,957 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for ba9cdc1b6bbd97ecf605751816de6a67 2023-05-02 02:55:32,957 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] compactions.Compactor(207): Compacting fdfb5f14b6794ceb90362e1e11a88edf.4b5519600ec1165492215f7e7e85c863, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1682996131382 2023-05-02 02:55:32,957 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] compactions.Compactor(207): Compacting 3e1e2cafee0643c6815088f36cad45af.4b5519600ec1165492215f7e7e85c863, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1682996131549 2023-05-02 02:55:32,957 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] compactions.Compactor(207): Compacting c030ba5f846346de9dea5f5911f62fe6.4b5519600ec1165492215f7e7e85c863, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1682996131549 2023-05-02 02:55:32,958 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] compactions.Compactor(207): Compacting 988f32ec168a465cb6ca4435884e9032.4b5519600ec1165492215f7e7e85c863, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1682996131727 2023-05-02 02:55:32,958 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] compactions.Compactor(207): Compacting 92d58d8357e44ee8901c47dbe21bc584.4b5519600ec1165492215f7e7e85c863, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1682996131727 2023-05-02 02:55:32,958 INFO [StoreOpener-ba9cdc1b6bbd97ecf605751816de6a67-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region ba9cdc1b6bbd97ecf605751816de6a67 2023-05-02 02:55:32,958 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] compactions.Compactor(207): Compacting 5c333cfab2c447638884f21583402bdd.4b5519600ec1165492215f7e7e85c863, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1682996131894 2023-05-02 02:55:32,959 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] compactions.Compactor(207): Compacting 0900e2b6e1284554a0808b2855de8862.4b5519600ec1165492215f7e7e85c863, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1682996131894 2023-05-02 02:55:32,959 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] compactions.Compactor(207): Compacting 12156fd9614f4ac59e717b6195f80411.4b5519600ec1165492215f7e7e85c863, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1682996132081 2023-05-02 02:55:32,959 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] compactions.Compactor(207): Compacting 1c974c743be54651beb4e762e81df9c3.4b5519600ec1165492215f7e7e85c863, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1682996132081 2023-05-02 02:55:32,959 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=150, resume processing ppid=148 2023-05-02 02:55:32,959 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=150, ppid=148, state=SUCCESS; OpenRegionProcedure 14a0d63ed7997d8b7bc0247c16a8acc1, server=jenkins-hbase4.apache.org,42019,1682996021676 in 227 msec 2023-05-02 02:55:32,959 DEBUG [StoreOpener-ba9cdc1b6bbd97ecf605751816de6a67-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf1 2023-05-02 02:55:32,960 DEBUG [StoreOpener-ba9cdc1b6bbd97ecf605751816de6a67-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf1 2023-05-02 02:55:32,960 INFO [StoreOpener-ba9cdc1b6bbd97ecf605751816de6a67-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ba9cdc1b6bbd97ecf605751816de6a67 columnFamilyName cf1 2023-05-02 02:55:32,961 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=148, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=14a0d63ed7997d8b7bc0247c16a8acc1, ASSIGN in 388 msec 2023-05-02 02:55:32,977 INFO [RS:2;jenkins-hbase4:42019-longCompactions-0] throttle.PressureAwareThroughputController(145): 14a0d63ed7997d8b7bc0247c16a8acc1#cf2#compaction#142 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-05-02 02:55:32,977 INFO [RS:2;jenkins-hbase4:42019-shortCompactions-0] throttle.PressureAwareThroughputController(145): 14a0d63ed7997d8b7bc0247c16a8acc1#cf1#compaction#143 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-02 02:55:32,982 DEBUG [StoreOpener-ba9cdc1b6bbd97ecf605751816de6a67-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf1/12156fd9614f4ac59e717b6195f80411.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/12156fd9614f4ac59e717b6195f80411-bottom 2023-05-02 02:55:32,989 DEBUG [StoreOpener-ba9cdc1b6bbd97ecf605751816de6a67-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf1/3e1e2cafee0643c6815088f36cad45af.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/3e1e2cafee0643c6815088f36cad45af-bottom 2023-05-02 02:55:32,995 DEBUG [StoreOpener-ba9cdc1b6bbd97ecf605751816de6a67-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf1/40c93f88f8df4f8e85dfd0fe36da6ea3.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/40c93f88f8df4f8e85dfd0fe36da6ea3-bottom 2023-05-02 02:55:32,998 DEBUG [StoreOpener-ba9cdc1b6bbd97ecf605751816de6a67-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf1/4e4d404e9d4a496499b0ada94af7a3ed.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/4e4d404e9d4a496499b0ada94af7a3ed-bottom 2023-05-02 02:55:33,009 DEBUG [StoreOpener-ba9cdc1b6bbd97ecf605751816de6a67-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf1/5c333cfab2c447638884f21583402bdd.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/5c333cfab2c447638884f21583402bdd-bottom 2023-05-02 02:55:33,010 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/.tmp/cf1/0c00eaf108004e349d0def78dd5399ff as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf1/0c00eaf108004e349d0def78dd5399ff 2023-05-02 02:55:33,012 DEBUG [StoreOpener-ba9cdc1b6bbd97ecf605751816de6a67-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf1/988f32ec168a465cb6ca4435884e9032.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/988f32ec168a465cb6ca4435884e9032-bottom 2023-05-02 02:55:33,013 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/.tmp/cf2/01877453d7a54401997406fc446a60e6 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf2/01877453d7a54401997406fc446a60e6 2023-05-02 02:55:33,017 INFO [RS:2;jenkins-hbase4:42019-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 14a0d63ed7997d8b7bc0247c16a8acc1/cf1 of 14a0d63ed7997d8b7bc0247c16a8acc1 into 0c00eaf108004e349d0def78dd5399ff(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-02 02:55:33,017 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 14a0d63ed7997d8b7bc0247c16a8acc1: 2023-05-02 02:55:33,017 INFO [RS:2;jenkins-hbase4:42019-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1682996132087.14a0d63ed7997d8b7bc0247c16a8acc1., storeName=14a0d63ed7997d8b7bc0247c16a8acc1/cf1, priority=9, startTime=1682996132953; duration=0sec 2023-05-02 02:55:33,017 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-02 02:55:33,018 INFO [RS:2;jenkins-hbase4:42019-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 14a0d63ed7997d8b7bc0247c16a8acc1/cf2 of 14a0d63ed7997d8b7bc0247c16a8acc1 into 01877453d7a54401997406fc446a60e6(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-02 02:55:33,018 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 14a0d63ed7997d8b7bc0247c16a8acc1: 2023-05-02 02:55:33,018 INFO [RS:2;jenkins-hbase4:42019-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1682996132087.14a0d63ed7997d8b7bc0247c16a8acc1., storeName=14a0d63ed7997d8b7bc0247c16a8acc1/cf2, priority=9, startTime=1682996132953; duration=0sec 2023-05-02 02:55:33,018 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-02 02:55:33,019 DEBUG [StoreOpener-ba9cdc1b6bbd97ecf605751816de6a67-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf1/a15ebfd300ef4d08a9526159dd2f49ab.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/a15ebfd300ef4d08a9526159dd2f49ab-bottom 2023-05-02 02:55:33,019 INFO [StoreOpener-ba9cdc1b6bbd97ecf605751816de6a67-1] regionserver.HStore(310): Store=ba9cdc1b6bbd97ecf605751816de6a67/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-02 02:55:33,019 INFO [StoreOpener-ba9cdc1b6bbd97ecf605751816de6a67-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region ba9cdc1b6bbd97ecf605751816de6a67 2023-05-02 02:55:33,020 DEBUG [StoreOpener-ba9cdc1b6bbd97ecf605751816de6a67-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf2 2023-05-02 02:55:33,020 DEBUG [StoreOpener-ba9cdc1b6bbd97ecf605751816de6a67-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf2 2023-05-02 02:55:33,020 INFO [StoreOpener-ba9cdc1b6bbd97ecf605751816de6a67-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ba9cdc1b6bbd97ecf605751816de6a67 columnFamilyName cf2 2023-05-02 02:55:33,031 DEBUG [StoreOpener-ba9cdc1b6bbd97ecf605751816de6a67-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf2/0900e2b6e1284554a0808b2855de8862.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/0900e2b6e1284554a0808b2855de8862-bottom 2023-05-02 02:55:33,034 DEBUG [StoreOpener-ba9cdc1b6bbd97ecf605751816de6a67-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf2/1c974c743be54651beb4e762e81df9c3.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/1c974c743be54651beb4e762e81df9c3-bottom 2023-05-02 02:55:33,036 DEBUG [StoreOpener-ba9cdc1b6bbd97ecf605751816de6a67-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf2/5e3792546819434d8bea5f8087d140de.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/5e3792546819434d8bea5f8087d140de-bottom 2023-05-02 02:55:33,039 DEBUG [StoreOpener-ba9cdc1b6bbd97ecf605751816de6a67-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf2/92d58d8357e44ee8901c47dbe21bc584.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/92d58d8357e44ee8901c47dbe21bc584-bottom 2023-05-02 02:55:33,042 DEBUG [StoreOpener-ba9cdc1b6bbd97ecf605751816de6a67-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf2/b4e5a86ddabc457190064c61898d51ca.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/b4e5a86ddabc457190064c61898d51ca-bottom 2023-05-02 02:55:33,044 DEBUG [StoreOpener-ba9cdc1b6bbd97ecf605751816de6a67-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf2/c030ba5f846346de9dea5f5911f62fe6.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/c030ba5f846346de9dea5f5911f62fe6-bottom 2023-05-02 02:55:33,047 DEBUG [StoreOpener-ba9cdc1b6bbd97ecf605751816de6a67-1] regionserver.HStore(539): loaded hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf2/fdfb5f14b6794ceb90362e1e11a88edf.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/fdfb5f14b6794ceb90362e1e11a88edf-bottom 2023-05-02 02:55:33,048 INFO [StoreOpener-ba9cdc1b6bbd97ecf605751816de6a67-1] regionserver.HStore(310): Store=ba9cdc1b6bbd97ecf605751816de6a67/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-02 02:55:33,048 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67 2023-05-02 02:55:33,049 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67 2023-05-02 02:55:33,050 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-02 02:55:33,051 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for ba9cdc1b6bbd97ecf605751816de6a67 2023-05-02 02:55:33,052 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened ba9cdc1b6bbd97ecf605751816de6a67; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70536961, jitterRate=0.05108262598514557}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-02 02:55:33,052 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for ba9cdc1b6bbd97ecf605751816de6a67: 2023-05-02 02:55:33,053 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,,1682996132087.ba9cdc1b6bbd97ecf605751816de6a67., pid=149, masterSystemTime=1682996132878 2023-05-02 02:55:33,053 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-02 02:55:33,054 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-02 02:55:33,054 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-02 02:55:33,054 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-02 02:55:33,054 INFO [RS:2;jenkins-hbase4:42019-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,,1682996132087.ba9cdc1b6bbd97ecf605751816de6a67. 2023-05-02 02:55:33,054 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] regionserver.HStore(1912): ba9cdc1b6bbd97ecf605751816de6a67/cf1 is initiating minor compaction (all files) 2023-05-02 02:55:33,054 INFO [RS:2;jenkins-hbase4:42019-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,,1682996132087.ba9cdc1b6bbd97ecf605751816de6a67. 2023-05-02 02:55:33,054 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] regionserver.HStore(1912): ba9cdc1b6bbd97ecf605751816de6a67/cf2 is initiating minor compaction (all files) 2023-05-02 02:55:33,054 INFO [RS:2;jenkins-hbase4:42019-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of ba9cdc1b6bbd97ecf605751816de6a67/cf1 in testSplitTableRegion,,1682996132087.ba9cdc1b6bbd97ecf605751816de6a67. 2023-05-02 02:55:33,054 INFO [RS:2;jenkins-hbase4:42019-longCompactions-0] regionserver.HRegion(2259): Starting compaction of ba9cdc1b6bbd97ecf605751816de6a67/cf2 in testSplitTableRegion,,1682996132087.ba9cdc1b6bbd97ecf605751816de6a67. 2023-05-02 02:55:33,055 INFO [RS:2;jenkins-hbase4:42019-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf1/40c93f88f8df4f8e85dfd0fe36da6ea3.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/40c93f88f8df4f8e85dfd0fe36da6ea3-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf1/a15ebfd300ef4d08a9526159dd2f49ab.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/a15ebfd300ef4d08a9526159dd2f49ab-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf1/4e4d404e9d4a496499b0ada94af7a3ed.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/4e4d404e9d4a496499b0ada94af7a3ed-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf1/3e1e2cafee0643c6815088f36cad45af.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/3e1e2cafee0643c6815088f36cad45af-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf1/988f32ec168a465cb6ca4435884e9032.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/988f32ec168a465cb6ca4435884e9032-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf1/5c333cfab2c447638884f21583402bdd.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/5c333cfab2c447638884f21583402bdd-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf1/12156fd9614f4ac59e717b6195f80411.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/12156fd9614f4ac59e717b6195f80411-bottom] into tmpdir=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/.tmp, totalSize=34.9 K 2023-05-02 02:55:33,055 INFO [RS:2;jenkins-hbase4:42019-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf2/b4e5a86ddabc457190064c61898d51ca.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/b4e5a86ddabc457190064c61898d51ca-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf2/5e3792546819434d8bea5f8087d140de.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/5e3792546819434d8bea5f8087d140de-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf2/fdfb5f14b6794ceb90362e1e11a88edf.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/fdfb5f14b6794ceb90362e1e11a88edf-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf2/c030ba5f846346de9dea5f5911f62fe6.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/c030ba5f846346de9dea5f5911f62fe6-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf2/92d58d8357e44ee8901c47dbe21bc584.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/92d58d8357e44ee8901c47dbe21bc584-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf2/0900e2b6e1284554a0808b2855de8862.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/0900e2b6e1284554a0808b2855de8862-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf2/1c974c743be54651beb4e762e81df9c3.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/1c974c743be54651beb4e762e81df9c3-bottom] into tmpdir=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/.tmp, totalSize=34.9 K 2023-05-02 02:55:33,055 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] compactions.Compactor(207): Compacting 40c93f88f8df4f8e85dfd0fe36da6ea3.4b5519600ec1165492215f7e7e85c863, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1682996130641 2023-05-02 02:55:33,056 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] compactions.Compactor(207): Compacting b4e5a86ddabc457190064c61898d51ca.4b5519600ec1165492215f7e7e85c863, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1682996130641 2023-05-02 02:55:33,056 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,,1682996132087.ba9cdc1b6bbd97ecf605751816de6a67. 2023-05-02 02:55:33,056 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegion,,1682996132087.ba9cdc1b6bbd97ecf605751816de6a67. 2023-05-02 02:55:33,056 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] compactions.Compactor(207): Compacting a15ebfd300ef4d08a9526159dd2f49ab.4b5519600ec1165492215f7e7e85c863, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1682996131209 2023-05-02 02:55:33,056 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] compactions.Compactor(207): Compacting 5e3792546819434d8bea5f8087d140de.4b5519600ec1165492215f7e7e85c863, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1682996131209 2023-05-02 02:55:33,056 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=ba9cdc1b6bbd97ecf605751816de6a67, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:33,056 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] compactions.Compactor(207): Compacting 4e4d404e9d4a496499b0ada94af7a3ed.4b5519600ec1165492215f7e7e85c863, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1682996131382 2023-05-02 02:55:33,056 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1682996132087.ba9cdc1b6bbd97ecf605751816de6a67.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1682996133056"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1682996133056"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1682996133056"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682996133056"}]},"ts":"1682996133056"} 2023-05-02 02:55:33,056 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] compactions.Compactor(207): Compacting fdfb5f14b6794ceb90362e1e11a88edf.4b5519600ec1165492215f7e7e85c863, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1682996131382 2023-05-02 02:55:33,057 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] compactions.Compactor(207): Compacting c030ba5f846346de9dea5f5911f62fe6.4b5519600ec1165492215f7e7e85c863, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1682996131549 2023-05-02 02:55:33,057 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] compactions.Compactor(207): Compacting 3e1e2cafee0643c6815088f36cad45af.4b5519600ec1165492215f7e7e85c863, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1682996131549 2023-05-02 02:55:33,057 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] compactions.Compactor(207): Compacting 92d58d8357e44ee8901c47dbe21bc584.4b5519600ec1165492215f7e7e85c863, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1682996131727 2023-05-02 02:55:33,057 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] compactions.Compactor(207): Compacting 988f32ec168a465cb6ca4435884e9032.4b5519600ec1165492215f7e7e85c863, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1682996131727 2023-05-02 02:55:33,057 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] compactions.Compactor(207): Compacting 0900e2b6e1284554a0808b2855de8862.4b5519600ec1165492215f7e7e85c863, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1682996131894 2023-05-02 02:55:33,058 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] compactions.Compactor(207): Compacting 5c333cfab2c447638884f21583402bdd.4b5519600ec1165492215f7e7e85c863, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1682996131894 2023-05-02 02:55:33,058 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] compactions.Compactor(207): Compacting 1c974c743be54651beb4e762e81df9c3.4b5519600ec1165492215f7e7e85c863, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1682996132081 2023-05-02 02:55:33,058 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] compactions.Compactor(207): Compacting 12156fd9614f4ac59e717b6195f80411.4b5519600ec1165492215f7e7e85c863, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1682996132081 2023-05-02 02:55:33,059 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=149, resume processing ppid=147 2023-05-02 02:55:33,059 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=149, ppid=147, state=SUCCESS; OpenRegionProcedure ba9cdc1b6bbd97ecf605751816de6a67, server=jenkins-hbase4.apache.org,42019,1682996021676 in 330 msec 2023-05-02 02:55:33,061 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=147, resume processing ppid=144 2023-05-02 02:55:33,061 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=147, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=ba9cdc1b6bbd97ecf605751816de6a67, ASSIGN in 488 msec 2023-05-02 02:55:33,062 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=144, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegion, parent=4b5519600ec1165492215f7e7e85c863, daughterA=ba9cdc1b6bbd97ecf605751816de6a67, daughterB=14a0d63ed7997d8b7bc0247c16a8acc1 in 974 msec 2023-05-02 02:55:33,068 INFO [RS:2;jenkins-hbase4:42019-longCompactions-0] throttle.PressureAwareThroughputController(145): ba9cdc1b6bbd97ecf605751816de6a67#cf2#compaction#145 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-02 02:55:33,068 INFO [RS:2;jenkins-hbase4:42019-shortCompactions-0] throttle.PressureAwareThroughputController(145): ba9cdc1b6bbd97ecf605751816de6a67#cf1#compaction#144 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-02 02:55:33,080 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/.tmp/cf2/1d8ed98e40b249d9b1003621ef0641f4 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf2/1d8ed98e40b249d9b1003621ef0641f4 2023-05-02 02:55:33,082 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/.tmp/cf1/3f6bb0fe054448ff83a4c59d6030d34a as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf1/3f6bb0fe054448ff83a4c59d6030d34a 2023-05-02 02:55:33,085 INFO [RS:2;jenkins-hbase4:42019-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in ba9cdc1b6bbd97ecf605751816de6a67/cf2 of ba9cdc1b6bbd97ecf605751816de6a67 into 1d8ed98e40b249d9b1003621ef0641f4(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-02 02:55:33,085 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for ba9cdc1b6bbd97ecf605751816de6a67: 2023-05-02 02:55:33,085 INFO [RS:2;jenkins-hbase4:42019-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1682996132087.ba9cdc1b6bbd97ecf605751816de6a67., storeName=ba9cdc1b6bbd97ecf605751816de6a67/cf2, priority=9, startTime=1682996133054; duration=0sec 2023-05-02 02:55:33,085 DEBUG [RS:2;jenkins-hbase4:42019-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-02 02:55:33,086 INFO [RS:2;jenkins-hbase4:42019-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in ba9cdc1b6bbd97ecf605751816de6a67/cf1 of ba9cdc1b6bbd97ecf605751816de6a67 into 3f6bb0fe054448ff83a4c59d6030d34a(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-02 02:55:33,086 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for ba9cdc1b6bbd97ecf605751816de6a67: 2023-05-02 02:55:33,086 INFO [RS:2;jenkins-hbase4:42019-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1682996132087.ba9cdc1b6bbd97ecf605751816de6a67., storeName=ba9cdc1b6bbd97ecf605751816de6a67/cf1, priority=9, startTime=1682996133053; duration=0sec 2023-05-02 02:55:33,087 DEBUG [RS:2;jenkins-hbase4:42019-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-02 02:55:33,089 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42019] ipc.CallRunner(144): callId: 1116 service: ClientService methodName: Scan size: 143 connection: 172.31.14.131:58782 deadline: 1682996193089, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegion,,1682996129638.4b5519600ec1165492215f7e7e85c863. is not online on jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:33,199 DEBUG [Listener at localhost/37119] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-05-02 02:55:33,206 INFO [Listener at localhost/37119] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-05-02 02:55:33,209 DEBUG [Listener at localhost/37119] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-05-02 02:55:33,216 WARN [Listener at localhost/37119] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-02 02:55:33,408 INFO [Listener at localhost/37119] client.HBaseAdmin$15(890): Started disable of testSplitTableRegion 2023-05-02 02:55:33,409 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testSplitTableRegion 2023-05-02 02:55:33,410 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure2.ProcedureExecutor(1029): Stored pid=151, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegion 2023-05-02 02:55:33,413 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-05-02 02:55:33,414 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682996133413"}]},"ts":"1682996133413"} 2023-05-02 02:55:33,415 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLING in hbase:meta 2023-05-02 02:55:33,416 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegion to state=DISABLING 2023-05-02 02:55:33,417 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=ba9cdc1b6bbd97ecf605751816de6a67, UNASSIGN}, {pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=14a0d63ed7997d8b7bc0247c16a8acc1, UNASSIGN}] 2023-05-02 02:55:33,418 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=14a0d63ed7997d8b7bc0247c16a8acc1, UNASSIGN 2023-05-02 02:55:33,419 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=14a0d63ed7997d8b7bc0247c16a8acc1, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:33,419 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1682996132087.14a0d63ed7997d8b7bc0247c16a8acc1.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1682996133419"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1682996133419"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682996133419"}]},"ts":"1682996133419"} 2023-05-02 02:55:33,420 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=154, ppid=153, state=RUNNABLE; CloseRegionProcedure 14a0d63ed7997d8b7bc0247c16a8acc1, server=jenkins-hbase4.apache.org,42019,1682996021676}] 2023-05-02 02:55:33,422 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=ba9cdc1b6bbd97ecf605751816de6a67, UNASSIGN 2023-05-02 02:55:33,422 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=ba9cdc1b6bbd97ecf605751816de6a67, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:33,422 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1682996132087.ba9cdc1b6bbd97ecf605751816de6a67.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1682996133422"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1682996133422"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682996133422"}]},"ts":"1682996133422"} 2023-05-02 02:55:33,427 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=155, ppid=152, state=RUNNABLE; CloseRegionProcedure ba9cdc1b6bbd97ecf605751816de6a67, server=jenkins-hbase4.apache.org,42019,1682996021676}] 2023-05-02 02:55:33,514 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-05-02 02:55:33,572 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 14a0d63ed7997d8b7bc0247c16a8acc1 2023-05-02 02:55:33,572 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 14a0d63ed7997d8b7bc0247c16a8acc1, disabling compactions & flushes 2023-05-02 02:55:33,572 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegion,41,1682996132087.14a0d63ed7997d8b7bc0247c16a8acc1. 2023-05-02 02:55:33,572 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegion,41,1682996132087.14a0d63ed7997d8b7bc0247c16a8acc1. 2023-05-02 02:55:33,572 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegion,41,1682996132087.14a0d63ed7997d8b7bc0247c16a8acc1. after waiting 0 ms 2023-05-02 02:55:33,572 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegion,41,1682996132087.14a0d63ed7997d8b7bc0247c16a8acc1. 2023-05-02 02:55:33,573 DEBUG [StoreCloser-testSplitTableRegion,41,1682996132087.14a0d63ed7997d8b7bc0247c16a8acc1.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf1/40c93f88f8df4f8e85dfd0fe36da6ea3.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/40c93f88f8df4f8e85dfd0fe36da6ea3-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf1/a15ebfd300ef4d08a9526159dd2f49ab.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/a15ebfd300ef4d08a9526159dd2f49ab-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf1/4e4d404e9d4a496499b0ada94af7a3ed.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/4e4d404e9d4a496499b0ada94af7a3ed-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf1/3e1e2cafee0643c6815088f36cad45af.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/3e1e2cafee0643c6815088f36cad45af-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf1/988f32ec168a465cb6ca4435884e9032.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/988f32ec168a465cb6ca4435884e9032-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf1/5c333cfab2c447638884f21583402bdd.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/5c333cfab2c447638884f21583402bdd-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf1/12156fd9614f4ac59e717b6195f80411.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/12156fd9614f4ac59e717b6195f80411-top] to archive 2023-05-02 02:55:33,574 DEBUG [StoreCloser-testSplitTableRegion,41,1682996132087.14a0d63ed7997d8b7bc0247c16a8acc1.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-02 02:55:33,575 DEBUG [StoreCloser-testSplitTableRegion,41,1682996132087.14a0d63ed7997d8b7bc0247c16a8acc1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf1/40c93f88f8df4f8e85dfd0fe36da6ea3.4b5519600ec1165492215f7e7e85c863 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf1/40c93f88f8df4f8e85dfd0fe36da6ea3.4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:33,576 DEBUG [StoreCloser-testSplitTableRegion,41,1682996132087.14a0d63ed7997d8b7bc0247c16a8acc1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf1/a15ebfd300ef4d08a9526159dd2f49ab.4b5519600ec1165492215f7e7e85c863 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf1/a15ebfd300ef4d08a9526159dd2f49ab.4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:33,578 DEBUG [StoreCloser-testSplitTableRegion,41,1682996132087.14a0d63ed7997d8b7bc0247c16a8acc1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf1/4e4d404e9d4a496499b0ada94af7a3ed.4b5519600ec1165492215f7e7e85c863 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf1/4e4d404e9d4a496499b0ada94af7a3ed.4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:33,579 DEBUG [StoreCloser-testSplitTableRegion,41,1682996132087.14a0d63ed7997d8b7bc0247c16a8acc1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf1/3e1e2cafee0643c6815088f36cad45af.4b5519600ec1165492215f7e7e85c863 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf1/3e1e2cafee0643c6815088f36cad45af.4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:33,580 DEBUG [StoreCloser-testSplitTableRegion,41,1682996132087.14a0d63ed7997d8b7bc0247c16a8acc1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf1/988f32ec168a465cb6ca4435884e9032.4b5519600ec1165492215f7e7e85c863 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf1/988f32ec168a465cb6ca4435884e9032.4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:33,581 DEBUG [StoreCloser-testSplitTableRegion,41,1682996132087.14a0d63ed7997d8b7bc0247c16a8acc1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf1/5c333cfab2c447638884f21583402bdd.4b5519600ec1165492215f7e7e85c863 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf1/5c333cfab2c447638884f21583402bdd.4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:33,582 DEBUG [StoreCloser-testSplitTableRegion,41,1682996132087.14a0d63ed7997d8b7bc0247c16a8acc1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf1/12156fd9614f4ac59e717b6195f80411.4b5519600ec1165492215f7e7e85c863 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf1/12156fd9614f4ac59e717b6195f80411.4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:33,583 DEBUG [StoreCloser-testSplitTableRegion,41,1682996132087.14a0d63ed7997d8b7bc0247c16a8acc1.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf2/b4e5a86ddabc457190064c61898d51ca.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/b4e5a86ddabc457190064c61898d51ca-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf2/5e3792546819434d8bea5f8087d140de.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/5e3792546819434d8bea5f8087d140de-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf2/fdfb5f14b6794ceb90362e1e11a88edf.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/fdfb5f14b6794ceb90362e1e11a88edf-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf2/c030ba5f846346de9dea5f5911f62fe6.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/c030ba5f846346de9dea5f5911f62fe6-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf2/92d58d8357e44ee8901c47dbe21bc584.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/92d58d8357e44ee8901c47dbe21bc584-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf2/0900e2b6e1284554a0808b2855de8862.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/0900e2b6e1284554a0808b2855de8862-top, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf2/1c974c743be54651beb4e762e81df9c3.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/1c974c743be54651beb4e762e81df9c3-top] to archive 2023-05-02 02:55:33,584 DEBUG [StoreCloser-testSplitTableRegion,41,1682996132087.14a0d63ed7997d8b7bc0247c16a8acc1.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-02 02:55:33,585 DEBUG [StoreCloser-testSplitTableRegion,41,1682996132087.14a0d63ed7997d8b7bc0247c16a8acc1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf2/b4e5a86ddabc457190064c61898d51ca.4b5519600ec1165492215f7e7e85c863 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf2/b4e5a86ddabc457190064c61898d51ca.4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:33,586 DEBUG [StoreCloser-testSplitTableRegion,41,1682996132087.14a0d63ed7997d8b7bc0247c16a8acc1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf2/5e3792546819434d8bea5f8087d140de.4b5519600ec1165492215f7e7e85c863 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf2/5e3792546819434d8bea5f8087d140de.4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:33,587 DEBUG [StoreCloser-testSplitTableRegion,41,1682996132087.14a0d63ed7997d8b7bc0247c16a8acc1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf2/fdfb5f14b6794ceb90362e1e11a88edf.4b5519600ec1165492215f7e7e85c863 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf2/fdfb5f14b6794ceb90362e1e11a88edf.4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:33,588 DEBUG [StoreCloser-testSplitTableRegion,41,1682996132087.14a0d63ed7997d8b7bc0247c16a8acc1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf2/c030ba5f846346de9dea5f5911f62fe6.4b5519600ec1165492215f7e7e85c863 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf2/c030ba5f846346de9dea5f5911f62fe6.4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:33,589 DEBUG [StoreCloser-testSplitTableRegion,41,1682996132087.14a0d63ed7997d8b7bc0247c16a8acc1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf2/92d58d8357e44ee8901c47dbe21bc584.4b5519600ec1165492215f7e7e85c863 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf2/92d58d8357e44ee8901c47dbe21bc584.4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:33,590 DEBUG [StoreCloser-testSplitTableRegion,41,1682996132087.14a0d63ed7997d8b7bc0247c16a8acc1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf2/0900e2b6e1284554a0808b2855de8862.4b5519600ec1165492215f7e7e85c863 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf2/0900e2b6e1284554a0808b2855de8862.4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:33,591 DEBUG [StoreCloser-testSplitTableRegion,41,1682996132087.14a0d63ed7997d8b7bc0247c16a8acc1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf2/1c974c743be54651beb4e762e81df9c3.4b5519600ec1165492215f7e7e85c863 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf2/1c974c743be54651beb4e762e81df9c3.4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:33,594 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-05-02 02:55:33,595 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-02 02:55:33,595 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegion,41,1682996132087.14a0d63ed7997d8b7bc0247c16a8acc1. 2023-05-02 02:55:33,595 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 14a0d63ed7997d8b7bc0247c16a8acc1: 2023-05-02 02:55:33,596 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 14a0d63ed7997d8b7bc0247c16a8acc1 2023-05-02 02:55:33,596 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close ba9cdc1b6bbd97ecf605751816de6a67 2023-05-02 02:55:33,597 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing ba9cdc1b6bbd97ecf605751816de6a67, disabling compactions & flushes 2023-05-02 02:55:33,597 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegion,,1682996132087.ba9cdc1b6bbd97ecf605751816de6a67. 2023-05-02 02:55:33,597 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegion,,1682996132087.ba9cdc1b6bbd97ecf605751816de6a67. 2023-05-02 02:55:33,597 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegion,,1682996132087.ba9cdc1b6bbd97ecf605751816de6a67. after waiting 0 ms 2023-05-02 02:55:33,597 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegion,,1682996132087.ba9cdc1b6bbd97ecf605751816de6a67. 2023-05-02 02:55:33,597 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=14a0d63ed7997d8b7bc0247c16a8acc1, regionState=CLOSED 2023-05-02 02:55:33,597 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,41,1682996132087.14a0d63ed7997d8b7bc0247c16a8acc1.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1682996133597"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682996133597"}]},"ts":"1682996133597"} 2023-05-02 02:55:33,598 DEBUG [StoreCloser-testSplitTableRegion,,1682996132087.ba9cdc1b6bbd97ecf605751816de6a67.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf1/40c93f88f8df4f8e85dfd0fe36da6ea3.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/40c93f88f8df4f8e85dfd0fe36da6ea3-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf1/a15ebfd300ef4d08a9526159dd2f49ab.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/a15ebfd300ef4d08a9526159dd2f49ab-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf1/4e4d404e9d4a496499b0ada94af7a3ed.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/4e4d404e9d4a496499b0ada94af7a3ed-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf1/3e1e2cafee0643c6815088f36cad45af.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/3e1e2cafee0643c6815088f36cad45af-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf1/988f32ec168a465cb6ca4435884e9032.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/988f32ec168a465cb6ca4435884e9032-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf1/5c333cfab2c447638884f21583402bdd.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/5c333cfab2c447638884f21583402bdd-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf1/12156fd9614f4ac59e717b6195f80411.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/12156fd9614f4ac59e717b6195f80411-bottom] to archive 2023-05-02 02:55:33,598 DEBUG [StoreCloser-testSplitTableRegion,,1682996132087.ba9cdc1b6bbd97ecf605751816de6a67.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-02 02:55:33,599 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=154, resume processing ppid=153 2023-05-02 02:55:33,600 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=154, ppid=153, state=SUCCESS; CloseRegionProcedure 14a0d63ed7997d8b7bc0247c16a8acc1, server=jenkins-hbase4.apache.org,42019,1682996021676 in 178 msec 2023-05-02 02:55:33,600 DEBUG [StoreCloser-testSplitTableRegion,,1682996132087.ba9cdc1b6bbd97ecf605751816de6a67.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf1/40c93f88f8df4f8e85dfd0fe36da6ea3.4b5519600ec1165492215f7e7e85c863 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf1/40c93f88f8df4f8e85dfd0fe36da6ea3.4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:33,601 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=153, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=14a0d63ed7997d8b7bc0247c16a8acc1, UNASSIGN in 183 msec 2023-05-02 02:55:33,601 DEBUG [StoreCloser-testSplitTableRegion,,1682996132087.ba9cdc1b6bbd97ecf605751816de6a67.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf1/a15ebfd300ef4d08a9526159dd2f49ab.4b5519600ec1165492215f7e7e85c863 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf1/a15ebfd300ef4d08a9526159dd2f49ab.4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:33,602 DEBUG [StoreCloser-testSplitTableRegion,,1682996132087.ba9cdc1b6bbd97ecf605751816de6a67.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf1/4e4d404e9d4a496499b0ada94af7a3ed.4b5519600ec1165492215f7e7e85c863 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf1/4e4d404e9d4a496499b0ada94af7a3ed.4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:33,603 DEBUG [StoreCloser-testSplitTableRegion,,1682996132087.ba9cdc1b6bbd97ecf605751816de6a67.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf1/3e1e2cafee0643c6815088f36cad45af.4b5519600ec1165492215f7e7e85c863 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf1/3e1e2cafee0643c6815088f36cad45af.4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:33,604 DEBUG [StoreCloser-testSplitTableRegion,,1682996132087.ba9cdc1b6bbd97ecf605751816de6a67.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf1/988f32ec168a465cb6ca4435884e9032.4b5519600ec1165492215f7e7e85c863 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf1/988f32ec168a465cb6ca4435884e9032.4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:33,605 DEBUG [StoreCloser-testSplitTableRegion,,1682996132087.ba9cdc1b6bbd97ecf605751816de6a67.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf1/5c333cfab2c447638884f21583402bdd.4b5519600ec1165492215f7e7e85c863 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf1/5c333cfab2c447638884f21583402bdd.4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:33,606 DEBUG [StoreCloser-testSplitTableRegion,,1682996132087.ba9cdc1b6bbd97ecf605751816de6a67.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf1/12156fd9614f4ac59e717b6195f80411.4b5519600ec1165492215f7e7e85c863 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf1/12156fd9614f4ac59e717b6195f80411.4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:33,607 DEBUG [StoreCloser-testSplitTableRegion,,1682996132087.ba9cdc1b6bbd97ecf605751816de6a67.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf2/b4e5a86ddabc457190064c61898d51ca.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/b4e5a86ddabc457190064c61898d51ca-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf2/5e3792546819434d8bea5f8087d140de.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/5e3792546819434d8bea5f8087d140de-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf2/fdfb5f14b6794ceb90362e1e11a88edf.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/fdfb5f14b6794ceb90362e1e11a88edf-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf2/c030ba5f846346de9dea5f5911f62fe6.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/c030ba5f846346de9dea5f5911f62fe6-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf2/92d58d8357e44ee8901c47dbe21bc584.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/92d58d8357e44ee8901c47dbe21bc584-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf2/0900e2b6e1284554a0808b2855de8862.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/0900e2b6e1284554a0808b2855de8862-bottom, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf2/1c974c743be54651beb4e762e81df9c3.4b5519600ec1165492215f7e7e85c863->hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/1c974c743be54651beb4e762e81df9c3-bottom] to archive 2023-05-02 02:55:33,608 DEBUG [StoreCloser-testSplitTableRegion,,1682996132087.ba9cdc1b6bbd97ecf605751816de6a67.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-02 02:55:33,619 DEBUG [StoreCloser-testSplitTableRegion,,1682996132087.ba9cdc1b6bbd97ecf605751816de6a67.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf2/b4e5a86ddabc457190064c61898d51ca.4b5519600ec1165492215f7e7e85c863 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf2/b4e5a86ddabc457190064c61898d51ca.4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:33,620 DEBUG [StoreCloser-testSplitTableRegion,,1682996132087.ba9cdc1b6bbd97ecf605751816de6a67.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf2/5e3792546819434d8bea5f8087d140de.4b5519600ec1165492215f7e7e85c863 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf2/5e3792546819434d8bea5f8087d140de.4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:33,621 DEBUG [StoreCloser-testSplitTableRegion,,1682996132087.ba9cdc1b6bbd97ecf605751816de6a67.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf2/fdfb5f14b6794ceb90362e1e11a88edf.4b5519600ec1165492215f7e7e85c863 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf2/fdfb5f14b6794ceb90362e1e11a88edf.4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:33,622 DEBUG [StoreCloser-testSplitTableRegion,,1682996132087.ba9cdc1b6bbd97ecf605751816de6a67.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf2/c030ba5f846346de9dea5f5911f62fe6.4b5519600ec1165492215f7e7e85c863 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf2/c030ba5f846346de9dea5f5911f62fe6.4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:33,623 DEBUG [StoreCloser-testSplitTableRegion,,1682996132087.ba9cdc1b6bbd97ecf605751816de6a67.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf2/92d58d8357e44ee8901c47dbe21bc584.4b5519600ec1165492215f7e7e85c863 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf2/92d58d8357e44ee8901c47dbe21bc584.4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:33,624 DEBUG [StoreCloser-testSplitTableRegion,,1682996132087.ba9cdc1b6bbd97ecf605751816de6a67.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf2/0900e2b6e1284554a0808b2855de8862.4b5519600ec1165492215f7e7e85c863 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf2/0900e2b6e1284554a0808b2855de8862.4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:33,625 DEBUG [StoreCloser-testSplitTableRegion,,1682996132087.ba9cdc1b6bbd97ecf605751816de6a67.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf2/1c974c743be54651beb4e762e81df9c3.4b5519600ec1165492215f7e7e85c863 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf2/1c974c743be54651beb4e762e81df9c3.4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:33,628 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-05-02 02:55:33,629 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-02 02:55:33,629 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegion,,1682996132087.ba9cdc1b6bbd97ecf605751816de6a67. 2023-05-02 02:55:33,629 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for ba9cdc1b6bbd97ecf605751816de6a67: 2023-05-02 02:55:33,630 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed ba9cdc1b6bbd97ecf605751816de6a67 2023-05-02 02:55:33,631 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=ba9cdc1b6bbd97ecf605751816de6a67, regionState=CLOSED 2023-05-02 02:55:33,631 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1682996132087.ba9cdc1b6bbd97ecf605751816de6a67.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1682996133630"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682996133630"}]},"ts":"1682996133630"} 2023-05-02 02:55:33,633 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=155, resume processing ppid=152 2023-05-02 02:55:33,633 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=155, ppid=152, state=SUCCESS; CloseRegionProcedure ba9cdc1b6bbd97ecf605751816de6a67, server=jenkins-hbase4.apache.org,42019,1682996021676 in 209 msec 2023-05-02 02:55:33,634 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=152, resume processing ppid=151 2023-05-02 02:55:33,634 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=152, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=ba9cdc1b6bbd97ecf605751816de6a67, UNASSIGN in 216 msec 2023-05-02 02:55:33,635 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682996133635"}]},"ts":"1682996133635"} 2023-05-02 02:55:33,635 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLED in hbase:meta 2023-05-02 02:55:33,637 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegion to state=DISABLED 2023-05-02 02:55:33,638 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=151, state=SUCCESS; DisableTableProcedure table=testSplitTableRegion in 228 msec 2023-05-02 02:55:33,715 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-05-02 02:55:33,715 INFO [Listener at localhost/37119] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegion, procId: 151 completed 2023-05-02 02:55:33,716 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testSplitTableRegion 2023-05-02 02:55:33,717 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] procedure2.ProcedureExecutor(1029): Stored pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegion 2023-05-02 02:55:33,719 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-05-02 02:55:33,719 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=156, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-05-02 02:55:33,719 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1227): Checking to see if procedure is done pid=156 2023-05-02 02:55:33,723 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:33,723 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1 2023-05-02 02:55:33,723 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67 2023-05-02 02:55:33,725 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1, FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2, FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/recovered.edits] 2023-05-02 02:55:33,726 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf1, FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf2, FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/recovered.edits] 2023-05-02 02:55:33,726 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf1, FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf2, FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/recovered.edits] 2023-05-02 02:55:33,728 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf1/0c00eaf108004e349d0def78dd5399ff to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf1/0c00eaf108004e349d0def78dd5399ff 2023-05-02 02:55:33,728 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf1/3f6bb0fe054448ff83a4c59d6030d34a to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf1/3f6bb0fe054448ff83a4c59d6030d34a 2023-05-02 02:55:33,729 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/12156fd9614f4ac59e717b6195f80411 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/12156fd9614f4ac59e717b6195f80411 2023-05-02 02:55:33,730 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/3e1e2cafee0643c6815088f36cad45af to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/3e1e2cafee0643c6815088f36cad45af 2023-05-02 02:55:33,731 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf2/1d8ed98e40b249d9b1003621ef0641f4 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/cf2/1d8ed98e40b249d9b1003621ef0641f4 2023-05-02 02:55:33,731 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf2/01877453d7a54401997406fc446a60e6 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/cf2/01877453d7a54401997406fc446a60e6 2023-05-02 02:55:33,732 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/40c93f88f8df4f8e85dfd0fe36da6ea3 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/40c93f88f8df4f8e85dfd0fe36da6ea3 2023-05-02 02:55:33,733 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/4e4d404e9d4a496499b0ada94af7a3ed to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/4e4d404e9d4a496499b0ada94af7a3ed 2023-05-02 02:55:33,733 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/recovered.edits/90.seqid to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67/recovered.edits/90.seqid 2023-05-02 02:55:33,734 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/recovered.edits/90.seqid to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1/recovered.edits/90.seqid 2023-05-02 02:55:33,734 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegion/ba9cdc1b6bbd97ecf605751816de6a67 2023-05-02 02:55:33,734 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegion/14a0d63ed7997d8b7bc0247c16a8acc1 2023-05-02 02:55:33,735 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/5c333cfab2c447638884f21583402bdd to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/5c333cfab2c447638884f21583402bdd 2023-05-02 02:55:33,736 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/988f32ec168a465cb6ca4435884e9032 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/988f32ec168a465cb6ca4435884e9032 2023-05-02 02:55:33,737 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/a15ebfd300ef4d08a9526159dd2f49ab to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf1/a15ebfd300ef4d08a9526159dd2f49ab 2023-05-02 02:55:33,739 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/0900e2b6e1284554a0808b2855de8862 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/0900e2b6e1284554a0808b2855de8862 2023-05-02 02:55:33,741 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/1c974c743be54651beb4e762e81df9c3 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/1c974c743be54651beb4e762e81df9c3 2023-05-02 02:55:33,742 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/5e3792546819434d8bea5f8087d140de to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/5e3792546819434d8bea5f8087d140de 2023-05-02 02:55:33,743 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/92d58d8357e44ee8901c47dbe21bc584 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/92d58d8357e44ee8901c47dbe21bc584 2023-05-02 02:55:33,744 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/b4e5a86ddabc457190064c61898d51ca to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/b4e5a86ddabc457190064c61898d51ca 2023-05-02 02:55:33,745 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/c030ba5f846346de9dea5f5911f62fe6 to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/c030ba5f846346de9dea5f5911f62fe6 2023-05-02 02:55:33,745 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/fdfb5f14b6794ceb90362e1e11a88edf to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/cf2/fdfb5f14b6794ceb90362e1e11a88edf 2023-05-02 02:55:33,747 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/recovered.edits/85.seqid to hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/archive/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863/recovered.edits/85.seqid 2023-05-02 02:55:33,748 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/.tmp/data/default/testSplitTableRegion/4b5519600ec1165492215f7e7e85c863 2023-05-02 02:55:33,748 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-05-02 02:55:33,750 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=156, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-05-02 02:55:33,751 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegion from hbase:meta 2023-05-02 02:55:33,753 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegion' descriptor. 2023-05-02 02:55:33,755 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=156, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-05-02 02:55:33,755 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegion' from region states. 2023-05-02 02:55:33,755 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1682996129638.4b5519600ec1165492215f7e7e85c863.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1682996133755"}]},"ts":"9223372036854775807"} 2023-05-02 02:55:33,755 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1682996132087.ba9cdc1b6bbd97ecf605751816de6a67.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1682996133755"}]},"ts":"9223372036854775807"} 2023-05-02 02:55:33,755 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,41,1682996132087.14a0d63ed7997d8b7bc0247c16a8acc1.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1682996133755"}]},"ts":"9223372036854775807"} 2023-05-02 02:55:33,757 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-05-02 02:55:33,757 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 4b5519600ec1165492215f7e7e85c863, NAME => 'testSplitTableRegion,,1682996129638.4b5519600ec1165492215f7e7e85c863.', STARTKEY => '', ENDKEY => ''}, {ENCODED => ba9cdc1b6bbd97ecf605751816de6a67, NAME => 'testSplitTableRegion,,1682996132087.ba9cdc1b6bbd97ecf605751816de6a67.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 14a0d63ed7997d8b7bc0247c16a8acc1, NAME => 'testSplitTableRegion,41,1682996132087.14a0d63ed7997d8b7bc0247c16a8acc1.', STARTKEY => '41', ENDKEY => ''}] 2023-05-02 02:55:33,757 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegion' as deleted. 2023-05-02 02:55:33,757 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1682996133757"}]},"ts":"9223372036854775807"} 2023-05-02 02:55:33,758 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegion state from META 2023-05-02 02:55:33,760 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=156, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-05-02 02:55:33,761 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=156, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegion in 44 msec 2023-05-02 02:55:33,820 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41119] master.MasterRpcServices(1227): Checking to see if procedure is done pid=156 2023-05-02 02:55:33,821 INFO [Listener at localhost/37119] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegion, procId: 156 completed 2023-05-02 02:55:33,836 INFO [Listener at localhost/37119] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=376 (was 375) - Thread LEAK? -, OpenFileDescriptor=844 (was 838) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=162 (was 159) - SystemLoadAverage LEAK? -, ProcessCount=170 (was 170), AvailableMemoryMB=13687 (was 13745) 2023-05-02 02:55:33,836 INFO [Listener at localhost/37119] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-05-02 02:55:33,836 INFO [Listener at localhost/37119] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-05-02 02:55:33,836 DEBUG [Listener at localhost/37119] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x51c18fa6 to 127.0.0.1:53331 2023-05-02 02:55:33,837 DEBUG [Listener at localhost/37119] ipc.AbstractRpcClient(494): Stopping rpc client 2023-05-02 02:55:33,837 DEBUG [Listener at localhost/37119] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-05-02 02:55:33,837 DEBUG [Listener at localhost/37119] util.JVMClusterUtil(257): Found active master hash=1914163511, stopped=false 2023-05-02 02:55:33,838 INFO [Listener at localhost/37119] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,41119,1682996020397 2023-05-02 02:55:33,845 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-05-02 02:55:33,845 INFO [Listener at localhost/37119] procedure2.ProcedureExecutor(629): Stopping 2023-05-02 02:55:33,845 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-05-02 02:55:33,845 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-05-02 02:55:33,845 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-05-02 02:55:33,845 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-05-02 02:55:33,845 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-05-02 02:55:33,845 DEBUG [Listener at localhost/37119] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x64a54fc5 to 127.0.0.1:53331 2023-05-02 02:55:33,846 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-05-02 02:55:33,846 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-05-02 02:55:33,846 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-05-02 02:55:33,846 DEBUG [Listener at localhost/37119] ipc.AbstractRpcClient(494): Stopping rpc client 2023-05-02 02:55:33,846 INFO [Listener at localhost/37119] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,39081,1682996021579' ***** 2023-05-02 02:55:33,846 INFO [Listener at localhost/37119] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-05-02 02:55:33,846 INFO [Listener at localhost/37119] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,38925,1682996021633' ***** 2023-05-02 02:55:33,846 INFO [Listener at localhost/37119] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-05-02 02:55:33,846 INFO [Listener at localhost/37119] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,42019,1682996021676' ***** 2023-05-02 02:55:33,846 INFO [Listener at localhost/37119] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-05-02 02:55:33,846 INFO [RS:1;jenkins-hbase4:38925] regionserver.HeapMemoryManager(220): Stopping 2023-05-02 02:55:33,847 INFO [RS:1;jenkins-hbase4:38925] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-05-02 02:55:33,847 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-05-02 02:55:33,847 INFO [RS:1;jenkins-hbase4:38925] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-05-02 02:55:33,847 INFO [RS:1;jenkins-hbase4:38925] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:33,847 DEBUG [RS:1;jenkins-hbase4:38925] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x25d1d127 to 127.0.0.1:53331 2023-05-02 02:55:33,847 DEBUG [RS:1;jenkins-hbase4:38925] ipc.AbstractRpcClient(494): Stopping rpc client 2023-05-02 02:55:33,847 INFO [RS:1;jenkins-hbase4:38925] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-05-02 02:55:33,847 INFO [RS:1;jenkins-hbase4:38925] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-05-02 02:55:33,847 INFO [RS:1;jenkins-hbase4:38925] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-05-02 02:55:33,847 INFO [RS:1;jenkins-hbase4:38925] regionserver.HRegionServer(3304): Received CLOSE for 1588230740 2023-05-02 02:55:33,848 INFO [RS:1;jenkins-hbase4:38925] regionserver.HRegionServer(1475): Waiting on 1 regions to close 2023-05-02 02:55:33,848 DEBUG [RS:1;jenkins-hbase4:38925] regionserver.HRegionServer(1479): Online Regions={1588230740=hbase:meta,,1.1588230740} 2023-05-02 02:55:33,848 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-05-02 02:55:33,848 INFO [RS:2;jenkins-hbase4:42019] regionserver.HeapMemoryManager(220): Stopping 2023-05-02 02:55:33,848 INFO [RS:0;jenkins-hbase4:39081] regionserver.HeapMemoryManager(220): Stopping 2023-05-02 02:55:33,848 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-05-02 02:55:33,848 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-05-02 02:55:33,848 INFO [RS:2;jenkins-hbase4:42019] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-05-02 02:55:33,848 INFO [RS:2;jenkins-hbase4:42019] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-05-02 02:55:33,848 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-05-02 02:55:33,848 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-05-02 02:55:33,848 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-05-02 02:55:33,848 INFO [RS:0;jenkins-hbase4:39081] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-05-02 02:55:33,848 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-05-02 02:55:33,849 INFO [RS:0;jenkins-hbase4:39081] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-05-02 02:55:33,849 INFO [RS:0;jenkins-hbase4:39081] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:33,848 INFO [RS:2;jenkins-hbase4:42019] regionserver.HRegionServer(3304): Received CLOSE for 5c823d6f15ad7b8fe82f9ae4beee6513 2023-05-02 02:55:33,849 DEBUG [RS:0;jenkins-hbase4:39081] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1fc1cff8 to 127.0.0.1:53331 2023-05-02 02:55:33,849 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=79.40 KB heapSize=125.77 KB 2023-05-02 02:55:33,849 DEBUG [RS:1;jenkins-hbase4:38925] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-05-02 02:55:33,849 DEBUG [RS:0;jenkins-hbase4:39081] ipc.AbstractRpcClient(494): Stopping rpc client 2023-05-02 02:55:33,849 INFO [RS:2;jenkins-hbase4:42019] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:33,849 INFO [RS:0;jenkins-hbase4:39081] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,39081,1682996021579; all regions closed. 2023-05-02 02:55:33,849 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 5c823d6f15ad7b8fe82f9ae4beee6513, disabling compactions & flushes 2023-05-02 02:55:33,849 DEBUG [RS:2;jenkins-hbase4:42019] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6cbc9d46 to 127.0.0.1:53331 2023-05-02 02:55:33,849 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1682996023643.5c823d6f15ad7b8fe82f9ae4beee6513. 2023-05-02 02:55:33,849 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1682996023643.5c823d6f15ad7b8fe82f9ae4beee6513. 2023-05-02 02:55:33,849 DEBUG [RS:2;jenkins-hbase4:42019] ipc.AbstractRpcClient(494): Stopping rpc client 2023-05-02 02:55:33,849 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1682996023643.5c823d6f15ad7b8fe82f9ae4beee6513. after waiting 0 ms 2023-05-02 02:55:33,849 INFO [RS:2;jenkins-hbase4:42019] regionserver.HRegionServer(1475): Waiting on 1 regions to close 2023-05-02 02:55:33,849 DEBUG [RS:2;jenkins-hbase4:42019] regionserver.HRegionServer(1479): Online Regions={5c823d6f15ad7b8fe82f9ae4beee6513=hbase:namespace,,1682996023643.5c823d6f15ad7b8fe82f9ae4beee6513.} 2023-05-02 02:55:33,850 DEBUG [RS:2;jenkins-hbase4:42019] regionserver.HRegionServer(1505): Waiting on 5c823d6f15ad7b8fe82f9ae4beee6513 2023-05-02 02:55:33,849 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1682996023643.5c823d6f15ad7b8fe82f9ae4beee6513. 2023-05-02 02:55:33,850 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 5c823d6f15ad7b8fe82f9ae4beee6513 1/1 column families, dataSize=78 B heapSize=488 B 2023-05-02 02:55:33,865 DEBUG [RS:0;jenkins-hbase4:39081] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/oldWALs 2023-05-02 02:55:33,865 INFO [RS:0;jenkins-hbase4:39081] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C39081%2C1682996021579:(num 1682996023201) 2023-05-02 02:55:33,865 DEBUG [RS:0;jenkins-hbase4:39081] ipc.AbstractRpcClient(494): Stopping rpc client 2023-05-02 02:55:33,865 INFO [RS:0;jenkins-hbase4:39081] regionserver.LeaseManager(133): Closed leases 2023-05-02 02:55:33,865 INFO [RS:0;jenkins-hbase4:39081] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-05-02 02:55:33,866 INFO [RS:0;jenkins-hbase4:39081] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-05-02 02:55:33,866 INFO [RS:0;jenkins-hbase4:39081] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-05-02 02:55:33,866 INFO [RS:0;jenkins-hbase4:39081] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-05-02 02:55:33,867 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-05-02 02:55:33,867 INFO [RS:0;jenkins-hbase4:39081] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:39081 2023-05-02 02:55:33,869 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/hbase/namespace/5c823d6f15ad7b8fe82f9ae4beee6513/.tmp/info/27a8f80ad6b14173835a828a376c2646 2023-05-02 02:55:33,873 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:33,873 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:33,874 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,39081,1682996021579 2023-05-02 02:55:33,874 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-05-02 02:55:33,874 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-05-02 02:55:33,874 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-05-02 02:55:33,873 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-05-02 02:55:33,874 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/hbase/namespace/5c823d6f15ad7b8fe82f9ae4beee6513/.tmp/info/27a8f80ad6b14173835a828a376c2646 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/hbase/namespace/5c823d6f15ad7b8fe82f9ae4beee6513/info/27a8f80ad6b14173835a828a376c2646 2023-05-02 02:55:33,874 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,39081,1682996021579] 2023-05-02 02:55:33,874 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,39081,1682996021579; numProcessing=1 2023-05-02 02:55:33,876 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,39081,1682996021579 already deleted, retry=false 2023-05-02 02:55:33,876 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,39081,1682996021579 expired; onlineServers=2 2023-05-02 02:55:33,878 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/hbase/namespace/5c823d6f15ad7b8fe82f9ae4beee6513/info/27a8f80ad6b14173835a828a376c2646, entries=2, sequenceid=6, filesize=4.8 K 2023-05-02 02:55:33,879 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 5c823d6f15ad7b8fe82f9ae4beee6513 in 29ms, sequenceid=6, compaction requested=false 2023-05-02 02:55:33,884 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=71.29 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/hbase/meta/1588230740/.tmp/info/38a3d389640a4437ad7fb5ab550f3bef 2023-05-02 02:55:33,887 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 38a3d389640a4437ad7fb5ab550f3bef 2023-05-02 02:55:33,904 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/hbase/namespace/5c823d6f15ad7b8fe82f9ae4beee6513/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-05-02 02:55:33,905 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-02 02:55:33,906 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1682996023643.5c823d6f15ad7b8fe82f9ae4beee6513. 2023-05-02 02:55:33,906 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 5c823d6f15ad7b8fe82f9ae4beee6513: 2023-05-02 02:55:33,906 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1682996023643.5c823d6f15ad7b8fe82f9ae4beee6513. 2023-05-02 02:55:33,906 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-05-02 02:55:33,906 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-05-02 02:55:33,906 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-05-02 02:55:33,907 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.58 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/hbase/meta/1588230740/.tmp/rep_barrier/db02281646c146ebbcdeed5ece578fa2 2023-05-02 02:55:33,911 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for db02281646c146ebbcdeed5ece578fa2 2023-05-02 02:55:33,923 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-05-02 02:55:33,926 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=5.53 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/hbase/meta/1588230740/.tmp/table/049d7989267c44fcaca6ab59a984eebe 2023-05-02 02:55:33,930 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 049d7989267c44fcaca6ab59a984eebe 2023-05-02 02:55:33,930 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/hbase/meta/1588230740/.tmp/info/38a3d389640a4437ad7fb5ab550f3bef as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/hbase/meta/1588230740/info/38a3d389640a4437ad7fb5ab550f3bef 2023-05-02 02:55:33,933 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 38a3d389640a4437ad7fb5ab550f3bef 2023-05-02 02:55:33,934 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/hbase/meta/1588230740/info/38a3d389640a4437ad7fb5ab550f3bef, entries=59, sequenceid=210, filesize=11.1 K 2023-05-02 02:55:33,934 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/hbase/meta/1588230740/.tmp/rep_barrier/db02281646c146ebbcdeed5ece578fa2 as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/hbase/meta/1588230740/rep_barrier/db02281646c146ebbcdeed5ece578fa2 2023-05-02 02:55:33,938 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for db02281646c146ebbcdeed5ece578fa2 2023-05-02 02:55:33,938 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/hbase/meta/1588230740/rep_barrier/db02281646c146ebbcdeed5ece578fa2, entries=24, sequenceid=210, filesize=7.5 K 2023-05-02 02:55:33,941 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/hbase/meta/1588230740/.tmp/table/049d7989267c44fcaca6ab59a984eebe as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/hbase/meta/1588230740/table/049d7989267c44fcaca6ab59a984eebe 2023-05-02 02:55:33,947 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 049d7989267c44fcaca6ab59a984eebe 2023-05-02 02:55:33,947 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/hbase/meta/1588230740/table/049d7989267c44fcaca6ab59a984eebe, entries=36, sequenceid=210, filesize=8.0 K 2023-05-02 02:55:33,948 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~79.40 KB/81308, heapSize ~125.73 KB/128744, currentSize=0 B/0 for 1588230740 in 99ms, sequenceid=210, compaction requested=false 2023-05-02 02:55:33,959 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/data/hbase/meta/1588230740/recovered.edits/213.seqid, newMaxSeqId=213, maxSeqId=1 2023-05-02 02:55:33,960 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-02 02:55:33,960 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-05-02 02:55:33,961 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-05-02 02:55:33,961 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-05-02 02:55:33,961 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-05-02 02:55:33,975 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-05-02 02:55:33,975 INFO [RS:0;jenkins-hbase4:39081] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,39081,1682996021579; zookeeper connection closed. 2023-05-02 02:55:33,975 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:39081-0x101dd492af80001, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-05-02 02:55:33,976 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@5368c541] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@5368c541 2023-05-02 02:55:34,049 INFO [RS:1;jenkins-hbase4:38925] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,38925,1682996021633; all regions closed. 2023-05-02 02:55:34,050 INFO [RS:2;jenkins-hbase4:42019] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,42019,1682996021676; all regions closed. 2023-05-02 02:55:34,053 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-05-02 02:55:34,053 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-05-02 02:55:34,057 DEBUG [RS:1;jenkins-hbase4:38925] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/oldWALs 2023-05-02 02:55:34,057 INFO [RS:1;jenkins-hbase4:38925] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C38925%2C1682996021633.meta:.meta(num 1682996023375) 2023-05-02 02:55:34,057 DEBUG [RS:2;jenkins-hbase4:42019] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/oldWALs 2023-05-02 02:55:34,057 INFO [RS:2;jenkins-hbase4:42019] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C42019%2C1682996021676:(num 1682996023201) 2023-05-02 02:55:34,057 DEBUG [RS:2;jenkins-hbase4:42019] ipc.AbstractRpcClient(494): Stopping rpc client 2023-05-02 02:55:34,057 INFO [RS:2;jenkins-hbase4:42019] regionserver.LeaseManager(133): Closed leases 2023-05-02 02:55:34,058 INFO [RS:2;jenkins-hbase4:42019] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-05-02 02:55:34,059 INFO [RS:2;jenkins-hbase4:42019] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-05-02 02:55:34,059 INFO [RS:2;jenkins-hbase4:42019] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-05-02 02:55:34,059 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-05-02 02:55:34,059 INFO [RS:2;jenkins-hbase4:42019] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-05-02 02:55:34,060 INFO [RS:2;jenkins-hbase4:42019] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:42019 2023-05-02 02:55:34,063 DEBUG [RS:1;jenkins-hbase4:38925] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/oldWALs 2023-05-02 02:55:34,063 INFO [RS:1;jenkins-hbase4:38925] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C38925%2C1682996021633:(num 1682996023201) 2023-05-02 02:55:34,063 DEBUG [RS:1;jenkins-hbase4:38925] ipc.AbstractRpcClient(494): Stopping rpc client 2023-05-02 02:55:34,063 INFO [RS:1;jenkins-hbase4:38925] regionserver.LeaseManager(133): Closed leases 2023-05-02 02:55:34,063 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:34,063 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-05-02 02:55:34,063 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,42019,1682996021676 2023-05-02 02:55:34,063 INFO [RS:1;jenkins-hbase4:38925] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-05-02 02:55:34,064 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-05-02 02:55:34,064 INFO [RS:1;jenkins-hbase4:38925] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:38925 2023-05-02 02:55:34,065 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,42019,1682996021676] 2023-05-02 02:55:34,065 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,42019,1682996021676; numProcessing=2 2023-05-02 02:55:34,066 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-05-02 02:55:34,066 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,38925,1682996021633 2023-05-02 02:55:34,066 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,42019,1682996021676 already deleted, retry=false 2023-05-02 02:55:34,066 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,42019,1682996021676 expired; onlineServers=1 2023-05-02 02:55:34,067 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,38925,1682996021633] 2023-05-02 02:55:34,067 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,38925,1682996021633; numProcessing=3 2023-05-02 02:55:34,069 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,38925,1682996021633 already deleted, retry=false 2023-05-02 02:55:34,069 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,38925,1682996021633 expired; onlineServers=0 2023-05-02 02:55:34,069 INFO [RegionServerTracker-0] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,41119,1682996020397' ***** 2023-05-02 02:55:34,069 INFO [RegionServerTracker-0] regionserver.HRegionServer(2310): STOPPED: Cluster shutdown set; onlineServer=0 2023-05-02 02:55:34,069 DEBUG [M:0;jenkins-hbase4:41119] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@332adb0c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-05-02 02:55:34,070 INFO [M:0;jenkins-hbase4:41119] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,41119,1682996020397 2023-05-02 02:55:34,070 INFO [M:0;jenkins-hbase4:41119] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,41119,1682996020397; all regions closed. 2023-05-02 02:55:34,070 DEBUG [M:0;jenkins-hbase4:41119] ipc.AbstractRpcClient(494): Stopping rpc client 2023-05-02 02:55:34,070 DEBUG [M:0;jenkins-hbase4:41119] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-05-02 02:55:34,070 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-05-02 02:55:34,070 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1682996022813] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1682996022813,5,FailOnTimeoutGroup] 2023-05-02 02:55:34,070 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1682996022814] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1682996022814,5,FailOnTimeoutGroup] 2023-05-02 02:55:34,070 DEBUG [M:0;jenkins-hbase4:41119] cleaner.HFileCleaner(317): Stopping file delete threads 2023-05-02 02:55:34,070 INFO [M:0;jenkins-hbase4:41119] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-05-02 02:55:34,070 INFO [M:0;jenkins-hbase4:41119] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-05-02 02:55:34,070 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-05-02 02:55:34,070 INFO [M:0;jenkins-hbase4:41119] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-05-02 02:55:34,070 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-05-02 02:55:34,071 DEBUG [M:0;jenkins-hbase4:41119] master.HMaster(1512): Stopping service threads 2023-05-02 02:55:34,071 INFO [M:0;jenkins-hbase4:41119] procedure2.RemoteProcedureDispatcher(118): Stopping procedure remote dispatcher 2023-05-02 02:55:34,071 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-05-02 02:55:34,071 ERROR [M:0;jenkins-hbase4:41119] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-13,5,PEWorkerGroup] Thread[HFileArchiver-14,5,PEWorkerGroup] Thread[HFileArchiver-15,5,PEWorkerGroup] Thread[HFileArchiver-16,5,PEWorkerGroup] 2023-05-02 02:55:34,071 INFO [M:0;jenkins-hbase4:41119] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-02 02:55:34,071 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-05-02 02:55:34,071 DEBUG [M:0;jenkins-hbase4:41119] zookeeper.ZKUtil(398): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-05-02 02:55:34,071 WARN [M:0;jenkins-hbase4:41119] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-05-02 02:55:34,072 INFO [M:0;jenkins-hbase4:41119] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-02 02:55:34,072 INFO [M:0;jenkins-hbase4:41119] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-05-02 02:55:34,072 DEBUG [M:0;jenkins-hbase4:41119] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-05-02 02:55:34,072 INFO [M:0;jenkins-hbase4:41119] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-05-02 02:55:34,072 DEBUG [M:0;jenkins-hbase4:41119] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-05-02 02:55:34,072 DEBUG [M:0;jenkins-hbase4:41119] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-05-02 02:55:34,072 DEBUG [M:0;jenkins-hbase4:41119] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-05-02 02:55:34,072 INFO [M:0;jenkins-hbase4:41119] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=607.94 KB heapSize=718.93 KB 2023-05-02 02:55:34,087 INFO [M:0;jenkins-hbase4:41119] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=607.94 KB at sequenceid=1282 (bloomFilter=true), to=hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/8646bdf05fbd4d03a84b0416b443c6fb 2023-05-02 02:55:34,090 INFO [M:0;jenkins-hbase4:41119] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8646bdf05fbd4d03a84b0416b443c6fb 2023-05-02 02:55:34,090 DEBUG [M:0;jenkins-hbase4:41119] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/8646bdf05fbd4d03a84b0416b443c6fb as hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/8646bdf05fbd4d03a84b0416b443c6fb 2023-05-02 02:55:34,093 INFO [M:0;jenkins-hbase4:41119] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8646bdf05fbd4d03a84b0416b443c6fb 2023-05-02 02:55:34,093 INFO [M:0;jenkins-hbase4:41119] regionserver.HStore(1080): Added hdfs://localhost:38649/user/jenkins/test-data/86c2d0c2-45ca-c042-ffb8-09f3dc7e1903/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/8646bdf05fbd4d03a84b0416b443c6fb, entries=156, sequenceid=1282, filesize=34.9 K 2023-05-02 02:55:34,094 INFO [M:0;jenkins-hbase4:41119] regionserver.HRegion(2948): Finished flush of dataSize ~607.94 KB/622526, heapSize ~718.91 KB/736168, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 22ms, sequenceid=1282, compaction requested=false 2023-05-02 02:55:34,095 INFO [M:0;jenkins-hbase4:41119] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-05-02 02:55:34,095 DEBUG [M:0;jenkins-hbase4:41119] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-05-02 02:55:34,099 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-05-02 02:55:34,099 INFO [M:0;jenkins-hbase4:41119] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-05-02 02:55:34,100 INFO [M:0;jenkins-hbase4:41119] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:41119 2023-05-02 02:55:34,102 DEBUG [M:0;jenkins-hbase4:41119] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,41119,1682996020397 already deleted, retry=false 2023-05-02 02:55:34,166 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-05-02 02:55:34,166 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:42019-0x101dd492af80003, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-05-02 02:55:34,166 INFO [RS:2;jenkins-hbase4:42019] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,42019,1682996021676; zookeeper connection closed. 2023-05-02 02:55:34,166 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@5800b726] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@5800b726 2023-05-02 02:55:34,213 ERROR [BP-376276558-172.31.14.131-1682996016523 heartbeating to localhost/127.0.0.1:38649] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-376276558-172.31.14.131-1682996016523 (Datanode Uuid 12e74f69-84cc-44e8-ac7e-14ebfb93f6fc) service to localhost/127.0.0.1:38649 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-05-02 02:55:34,266 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-05-02 02:55:34,266 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): master:41119-0x101dd492af80000, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-05-02 02:55:34,266 INFO [M:0;jenkins-hbase4:41119] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,41119,1682996020397; zookeeper connection closed. 2023-05-02 02:55:34,366 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-05-02 02:55:34,367 DEBUG [Listener at localhost/37119-EventThread] zookeeper.ZKWatcher(600): regionserver:38925-0x101dd492af80002, quorum=127.0.0.1:53331, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-05-02 02:55:34,367 INFO [RS:1;jenkins-hbase4:38925] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,38925,1682996021633; zookeeper connection closed. 2023-05-02 02:55:34,367 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@1923c950] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@1923c950 2023-05-02 02:55:34,367 INFO [Listener at localhost/37119] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 3 regionserver(s) complete 2023-05-02 02:55:34,368 WARN [Listener at localhost/37119] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-05-02 02:55:34,372 INFO [Listener at localhost/37119] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-05-02 02:55:34,479 WARN [BP-376276558-172.31.14.131-1682996016523 heartbeating to localhost/127.0.0.1:38649] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-05-02 02:55:34,479 WARN [BP-376276558-172.31.14.131-1682996016523 heartbeating to localhost/127.0.0.1:38649] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-376276558-172.31.14.131-1682996016523 (Datanode Uuid e5b41dfb-e591-48a4-b02d-848c5b0dba36) service to localhost/127.0.0.1:38649 2023-05-02 02:55:34,482 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e48d2209-986f-dd5e-f2e4-019027d7bcac/cluster_9b21c39f-0840-34d4-ae73-45b092a6b580/dfs/data/data5/current/BP-376276558-172.31.14.131-1682996016523] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-05-02 02:55:34,482 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e48d2209-986f-dd5e-f2e4-019027d7bcac/cluster_9b21c39f-0840-34d4-ae73-45b092a6b580/dfs/data/data6/current/BP-376276558-172.31.14.131-1682996016523] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-05-02 02:55:34,484 WARN [Listener at localhost/37119] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-05-02 02:55:34,487 INFO [Listener at localhost/37119] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-05-02 02:55:34,589 WARN [BP-376276558-172.31.14.131-1682996016523 heartbeating to localhost/127.0.0.1:38649] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-376276558-172.31.14.131-1682996016523 (Datanode Uuid 12e74f69-84cc-44e8-ac7e-14ebfb93f6fc) service to localhost/127.0.0.1:38649 2023-05-02 02:55:34,590 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e48d2209-986f-dd5e-f2e4-019027d7bcac/cluster_9b21c39f-0840-34d4-ae73-45b092a6b580/dfs/data/data3/current/BP-376276558-172.31.14.131-1682996016523] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-05-02 02:55:34,590 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e48d2209-986f-dd5e-f2e4-019027d7bcac/cluster_9b21c39f-0840-34d4-ae73-45b092a6b580/dfs/data/data4/current/BP-376276558-172.31.14.131-1682996016523] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-05-02 02:55:34,592 WARN [Listener at localhost/37119] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-05-02 02:55:34,594 INFO [Listener at localhost/37119] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-05-02 02:55:34,699 WARN [BP-376276558-172.31.14.131-1682996016523 heartbeating to localhost/127.0.0.1:38649] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-05-02 02:55:34,700 WARN [BP-376276558-172.31.14.131-1682996016523 heartbeating to localhost/127.0.0.1:38649] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-376276558-172.31.14.131-1682996016523 (Datanode Uuid 2b091f00-29ea-476e-ba4f-8ff3dee54078) service to localhost/127.0.0.1:38649 2023-05-02 02:55:34,701 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e48d2209-986f-dd5e-f2e4-019027d7bcac/cluster_9b21c39f-0840-34d4-ae73-45b092a6b580/dfs/data/data1/current/BP-376276558-172.31.14.131-1682996016523] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-05-02 02:55:34,701 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e48d2209-986f-dd5e-f2e4-019027d7bcac/cluster_9b21c39f-0840-34d4-ae73-45b092a6b580/dfs/data/data2/current/BP-376276558-172.31.14.131-1682996016523] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-05-02 02:55:34,726 INFO [Listener at localhost/37119] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-05-02 02:55:34,921 INFO [Listener at localhost/37119] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-05-02 02:55:35,160 INFO [Listener at localhost/37119] hbase.HBaseTestingUtility(1293): Minicluster is down